I think the referrer header was originally meant for internal redirects within a website. Like you click on the add comment button but you're not signed in, and after signing in the website can use the referrer to bring you back to the page you were on. Also useful for old school HTML form submissions that take you to the submission endpoint and then has to bring you back.

But early browser devs just didn't have cross site isolation in mind (you can tell because when JS first came out, you could literally use it to take control of an iframe loaded by your page regardless of what domain the iframe was on) so tracking companies took advantage of that and now they think they're entitled to it continuing to work that way because "it was always like that."