Keep at it, if curating All is what you want. That's how I use this site too, and so far I've blocked over 900 communities - it has made All SO MUCH better.
You'll burn through the redundant communities.
Just don't try to do it all in one go - hit a few on each scroll and it'll add up without being a very noticeable effort.