If you're not quite sure about getting into coffee, you can get started with a small budget, and you can make a nice cup of coffee that way. Probabaly not a great cup, but a cup of coffee you'll find enjoyable at the time. Once you start experimenting with different variables and digging a bit deeper into different flavors, you may notice that you're not getting the same cup every time. That's when you start pushing the limit of what's possible with the equipment you have available, and you'll realize that using a cheap grinder is sort of like making you play this game in the hard mode.
Examples: Using a cheap blender type grinder (blade grinder) means you can easily chop coffee to some unknown random particle size. You don’t pay much, but at least you can use recently roasted whole bean coffee, which is great. If you want to adjust the particle size in a specific way, that’s when it gets very tricky. Did you grind one second longer than last time? Too bad, now it’s way too fine and you’re getting a bitter cup as a result. Fortunately, you can easily fix that with milk, but as you start noticing more details in the taste profile, you start demanding more and more. Consequently, fixing mistakes with milk won’t be as appealing as it once was.
Using a cheap hand grinder is a lot better than an electric blender. Once you set the screw at a specific position, you’ll get the same particle size every time, which is great for consistency. What if you decide to use a french press today, but tomorrow you want to switch back to pour over, moka pot or AeroPress? Too bad, the grinder has no markings on it, so you’ll just have to eye-ball the setting and hope for the best. That’s obviously easier than timing your blender perfectly every time, but it’s still not exactly easy to get great results.
The solution: Eventually you’ll want to buy a burr grinder (electric or manual) with clear markings for different grind sizes.
When to invest? Don’t spend any more money on better gear until you’ve already “needed” better gear a few times in order to fix something you’ve noticed in your cup. As long as you’re relatively happy with your current gear and the cup it produces, there’s no need to spend more on this hobby. However, when you start noticing new things, that’s the right time to buy something that really addresses a specific thing you have in mind.
Summary: Get started with cheap gear and upgrade only when you think it’s necessary. Getting some random cup of coffee using cheap gear is easy, but getting a specific kind of cup with that gear is hard. Using more expensive gear will make it easier.
Something else? This is based on my personal experiences, and your experiences may differ. This rule of thumb might apply to grinders, but it is not guaranteed to apply to other types of equipment. For instance, I don’t own an espresso machine, so I have no idea how price is reflected on the end result in that case.