Brother, have you heard of buses? Even INSIDE cpus/socs bus speeds are a limitation. Also i fucking hate how the first thing people mention now is how ai could benefit from a jump in computing power.
Edit: I havent dabbled that much in high speed stuff yet but isnt the picosecond range so fast that the capacitance of simple traces and connectors between chips influence the rising and falling edge of chips?