My speculation:

The cpu chiplets only have the cpu cores and a presumably 3 port infinty fabric switch. There is no memory clock on the cpu die, so it wouldn’t make much sense for the switch on the cpu chiplet to run at a different clock from the cpu cores. I would expect the switch to just run at core clock, so the connection between the two CCXs on each chiplet could be extremely fast. With low latency between the two on die CCXs, you should have good latency between 8 cores and 32 MB of L3 on die. Also, a 3-port switch at core clock is probably very low latency. It will still be a latency hit to go to the other chiplet for the 12 core and 16 core packages, but with infinity fabric running at ridiculously high clock, and possibly around 50 GB/s, I suspect the latency to the other chiplet to be quite good also.

AFAIK, Zen 1 uses 128-bit in each direction internally. This makes a lot of sense since a 64-bit DDR memory controller can deliver 128-bits per memory clock. The IFOP links are then essentially 1/4 the width at about 4x the clock and the IFIS links are essentially 1/8 the width at essentially 8x the memory clock. Everything matches nicely. They more than doubled the infinity fabric speed for Zen 2 though. Doubling the clock speed would probably take a lot more power so doubling the internal width to 256-bit may be more likely. That may explain why the Epyc Rome IO die is so large if internal paths are double the width compared to Zen 1. The memory clock is going to be higher but it also may be more likely that they will combine the two memory channels into essentially a single 128-bit DDR channel to provide the 256-bits per clock minimum. The diagrams for Zen 1 generally show 2 independent IF ports for the two memory controllers; see the ISSCC Zeppelin slides.

Even for local memory accesses, the data still needs to go through the fabric switch in Zen 1. The switch in Zen 1 has a lot of ports which may make it quite large, even at 128-bit width. The ISSCC slides show 11 ports on the fabric switch (2 CCX, 2 memory controllers, 4 IFOP, 2 IFIS, and 1 IOMS). The one in Ryzen 3000 may only be 5 or 6 ports. It would have 2 CCX (IFOP), 2 pci express (IFIS), and possibly just 1 memory controller port at 128-bits wide. The IO die for AM4 could also be simplified a bit since it doesn’t need to connect to other cpus. The IFIS style links could be pci express IO only. So it may be twice the width, it may have a lot less ports, and it may be designed to operate at a lot higher clock speed since much higher clock speed memory is supported. That all looks good for latency. It does have an extra off chiplet link to go through for local memory compared to Zen 1, but the infinity fabric link speed is very high, so latency of that link is probably quite low. Also, If they double the internal switch width, then It can essentially transfer cache lines in half the time.

I don’t know what they will do about ThreadRipper. The 16-core AM4 Ryzen 3000 part will probably outperform 16-core ThreadRipper except for possibly extremely memory bound applications. Zen 2 can probably support higher memory clock, so it may still do quite well though. This leaves ThreadRipper owners without a cpu upgrade. They could release a new ThreadRipper later, but it is probably very low priority compared to Rome.