Many distro’s add kernel patches and add different drivers to the initrd.
Also the core os ( most minimal installation ) has many different tools and libs.
Also at time of release they can pick from many different versions of a single package.
That in combination with what version of GCC and compile flags can and does make a huge differance.