The poll by the ARD public broadcaster said 21% of respondents agreed with the proposition.
"It is racist. I feel we need to wake up. Many people in Europe had to flee.. searching for a safe country," Nagelsmann said on Sunday.
The 36-year-old said he agreed with Germany midfielder Joshua Kimmich, who described the questionnaire as "racist" a day earlier.
“Josh [Kimmich] responded really well, with a very clear and thought-out statement,” Nagelsmann said at a briefing at his team's training base.
“I see this in exactly the same way. This question is insane.”
“There are people in Europe who’ve had to flee because of war, economic factors, environmental disasters, people who simply want to be taken in," he went on.
"We have to ask what are we doing at the moment? We in Germany are doing very, very well, and when we say something like that, I think it’s crazy how we turn a blind eye and simply block out such things."
ARD - the German public broadcaster - said it had commissioned the survey to have measurable data, after a reporter working on a documentary on football and diversity was repeatedly asked about the make-up of the national team.
The poll was conducted among 1,304 randomly selected respondents.
Karl Valks, sports director with the ARD station who commissioned the poll, said the company was "dismayed that the results are what they are, but they are also an expression of the social situation in Germany today".
"Sport plays an important role in our society, the national team is a strong example of integration," German media cited him as saying.
The current national squad has a number of players with mixed heritage, including captain Ilkay Gündogan and winger Leroy Sané.
Germany is hosting the Euro 2024 tournament later this month, and Nagelsmann said his team would be playing "for everyone in the country". They will kick-off the competition with a clash against Scotland at Munich's Allianz Arena on 14 June.
The controversy comes just weeks after the team's kit manufacturer, Adidas, was forced to ban fans from buying German football kits customised with the number 44, after media raised their resemblance to the symbol used by World War Two-era Nazi SS units.
The SS was responsible for many of the crimes against humanity committed by the Nazis. Members of the SS ranged from Gestapo agents to concentration camp guards. SS duties included administering death camps where millions of Jews and others were put to death.
