>>55142036
It's mostly USE flags and kernel options. Arch packages are also compiled with -O2, but they do also get hardening by default which might make them somewhat slower too.
If you really want to try out ricey CFLAGS, do:
CFLAGS="-flto -O2 -march=native -pipe -fwhole-program -fuse-linker-plugin -ffat-lto-objects -fvisibility=hidden -floop-interchange -floop-strip-mine -floop-block -floop-nest-optimize -floop-parallelize-all -floop-parallelize-loops=4"
USE="graphite openmp cloop ppl"
LDFLAGS="-lgomp"
The last 4 CFLAGS can cause compilation issues, the others should be safe.
You will have to recompile gcc once or twice for cloop and graphite. There are more specific instructions in the gentoo wiki.
It's also a good idea to use a grsecurity kernel and USE="hardened".