Wrong, actually, if you trust internal CIA reports.
Anecdotes, especially familial ones, are not a replacement for expansive data taking. I have no idea what your great-grandfather was sent to prison for, nor is a single case like that representative of the entire USSR.
The Hungarian revolt in 1956 was infested with anti-semetic pograms. MI6 funded, supplied, and trained the Hungarian counter-revolutionaries. These counter-revolutionaries were allied with fascists who were lynching Jewish people and Communists.

"The special correspondent of the Yugoslav paper, Politika, (Nov. 13, 1956) describing the events of those days, said that the homes of Communists were marked with a white cross and those of Jews with a black cross, to serve as signs for the extermination squads. “There is no longer any room for doubt,” said the Yugoslav reporter, “it is an example of classic Hungarian fascism and of White Terror. The information,” continued this writer, "coming from the provinces tells how in certain places Communists were having their eyes put out, their ears cut off, and that they were being killed in the most terrible ways."

"But the forces of reaction were rapidly consolidating their power and pushing forward on the top levels, while in the streets the blood of scores of massacred Communists, Jews, and progressives was flowing."

"Some of the reports reaching Warsaw from Budapest today caused considerable concern. These reports told of massacres of Communists and Jews by what were described as 'Fascist elements' ...." (N.Y. Times, Nov. 1. 1956)

"The evidence is conclusive that the entry of Soviet troops into Budapest stopped the execution of scores, perhaps thousands of Jews, for by the end of October and early November, anti-Semtic pogroms - hallmark of unbridled fascistic terror - were making their appearance, after an absence of some ten years, within Hungary."

"A correspondent of the Israeli newspaper Maariv (Tel Aviv) reported:

During the uprising a number of former Nazis were released from prison and other former Nazis came to Hungary from Salzburg . . . I met them at the border . . . I saw anti-Semitic posters in Budapest . . . On the walls, street lights, streetcars, you saw inscriptions reading: “Down with Jew Gero!” “Down with Jew Rakosi!” or just simply “down with the Jews!”

Leading rabbinical circles in New York received a cable early in November from corresponding circles in Vienna that “Jewish blood is being shed by the rebels in Hungary.” Very much later-in February, 1957-the World Jewish Congress reported that “anti-Semitic excesses occurred in more than twenty villages and smaller provincial towns during the October-November revolt.” This occurred, according to this very conservative body, because “fascist and anti-Semitic groups had apparently seized the opportunity, presented by the absence of a central authority, to come to the surface.” Many among the Jewish refugees from Hungary, the report continued, had fled from this anti-Semitic pogrom-like atmosphere (N.Y. Times, Feb. 15, 1957). This confirmed the earlier report made by the British Rabbi, R. Pozner, who, after touring refugee camps, declared that “the majority of Jews who left Hungary did so for fear of the Hungarians and not the Russians.” The Paris Jewish newspaper, Naye Presse, asserted that Jewish refugees in France claimed quite generally that Soviet soldiers had saved their lives."

Further, the CIA also backed Hungarian resistance forces:

Prague in 1968 was a similar fascist uprising in both cases there were some elements of progressive protest, but these were greatly overshadowed by the fascist movements.

I'm not making any accusations here, I want you to elaborate more, but legitimately it sounds like you're saying your family members were fascists or fascist sympathizers. I want you to clear their names, because Hungary absolutely fought on the side of the Axis in World War II, and the 1956 counter-revolt was against the Communists.