Gonna take a controversial tack here:
No, they shouldn't be banned.
You have a choice to buy into an HOA or not. You can still buy plots of land, and build to suit; you are not obligated or limited solely to houses that already exist in HOAs. Yes, your up-front costs will be higher, because you're going to have to pay for putting in a well, septic system, and possibly running electric to your place, and if you want gas you'll need propane deliveries, versus hooking in to an existing water/sewer/electric/natural gas system. But that's still your choice.
Some people want HOAs because them come with amenities that people wouldn't have otherwise. In my area, there are two very large HOAs that both offer things like full golf courses, and horse stables with miles of trails for riding; the people buying in those HOAs buy into them because of those things; having enough land on your own to keep and ride horses is, well, good luck finding that much land as a single parcel within reasonable driving distance of any city.
Finally, if you buy a home, most of the time you want to know that your home isn't going to plummet in value. You're dropping a LOT of money on one, and hoping that, if you ever need to move, you'll be able to get the money you put in back out again. When you don't live in an HOA, there's always a risk that the things shitty neighbors will do will end up wrecking your property values. E.g., if the person right next to me starts parking dead cars in their driveway, and has yard dogs that are barking at all hours of the night, not many people are going to willingly move in next to them.
Do I LIKE HOAs? Not really. The best HOAs are the ones that have absolutely minimal interference in your daily life. I live in an HOA; they keep the road functional, specify certain aspects of new construction (minimum size--no tiny homes--colors have to be earth-tones, etc.) and... That's about it. Yeah, I'm supposed to get permission before I put in any yard statues more than 6' tall, and I'm not allowed to clear cut the trees on my property (...not that I ever plan to...), I can't put a pistol range in my yard and practice shooting at home, but that's about all. As long as I stay quiet, and mind my own business, the HOA doesn't give a shit. The most high-handed thing they've done in the last decade was amend the by-laws to ban short-term rentals; one shithead was renting out their place as an AirBnB, which led to loud parties on weekends and a lot of extra traffic.