This is a copy and past from my reply another community, sorry if you are reading it again:
I’m at the completely opposite end of the spectrum of most people, they should be public to all. It makes it clear whether the guy downvoting you is doing so maliciously or as a non-participant. Same for upvotes. Otherwise, just get rid of it and find some better mechanism. The people saying “NO!” or that they should be anonymous don’t really have a reason, your comment history is already giving you away and no one has a problem with that.
The worst thing public upvotes/downvotes might lead to are the same things your comments are already profiled for by the same people that would and perhaps a random getting mad at your downvote or upvote and voting back, which doesn’t matter that much with the current karma system. The benefits, however, are a clear vision of where those upvotes and downvotes are coming from, without it you are a blind person in a social networks but with it you can tell who is interacting with you and you can investigate why and even make judgement calls because you can see whether they interact like a jerk.
No drama witch hunts, accountability for the way you are interacting online, the the benefits outweighs the drawbacks, but people don’t want it because they feel insecure about it. I specially favor it because it could be a first step for a form of crowdsourced moderation (speculated on it here), where you can choose the people you think are voting comments to your taste to eventually have a select group large enough to determine which should show up first and which shouldn’t show at all, and it could be completely complementary to existing systems. Don’t want to see “yes, I agree” comments sorting as the most relevant? You might choose people who do not upvote but have engaged with the rest of the thread for comments you consider more informative.
No one from kbin/mbin instances can check out the downvotes you make, since this attitude has been so widespread many don’t report it to those instances. They can see people who upvote, and the sky hasn’t fallen because of it. Anonymity largely only helps the minority making the drama remain hidden.