Despite having a squad including the likes of Gareth Bale, Aaron Ramsey, Joe Allen, Ashley Williams and Bellamy himself, attendances at Wales internationals have dwindled in recent years.
Wednesday's 2-1 friendly victory over Austria in Swansea was watched by a crowd of just over 8,000, despite an adult ticket costing as little as £10.
That is in stark contrast to the 20,000 who regularly watch Premier League games at the ground, and the attendances at Championship leaders Cardiff.
It was only a decade ago that Wales attracted a 74,000 sell-out at the Millennium Stadium for a Euro 2004 qualifier against minnows Azerbaijan, as Mark Hughes' side launched a promising bid to reach the finals which ended in a heartbreaking defeat to Russia.
Bellamy was part of that qualifying campaign, and feels it will take repeats of the wins over the likes of Italy and Germany from Hughes' time in charge for fans to return to the national side, and see stars like Bale in the flesh.
"We are used to that," the Cardiff striker said of the turnout against the Austrians.
"We had Ryan Giggs playing in front of the same sort of attendances.
"The better the team is, the better the attendances we will have. More people will get to see Gareth Bale then.
"Winning helps, whether they're friendlies or qualifiers. If you don't reach that benchmark you don't get the attendances.
"It's a results business. The crowds are here, Cardiff and Swansea prove that, we do get the numbers following us. And with Swansea and Cardiff being successful, we need to get the national team playing at the same level and we will get the attendances.
"I remember growing up watching Cardiff and there were 1,500 fans, because Cardiff weren't good enough and were in the old fourth division.
"If you notice over the years, with Swansea as well, that attendances have gradually improved and people support you if you are successful.
"If we can get the international team back to the level I believe these players are capable of, and the glory days from when I started off and we had 70,000 sell-outs at the Millennium, if we do well enough and are successful the fans will come back."
Victory over Austria at least gives Wales valuable confidence and momentum ahead of next month's qualifiers against Scotland and Croatia.
Bellamy, who played his first international since August, hopes Chris Coleman's side can build on their performance.
"That's why it was so important for us to win," he said. "It sets us up and gives us the confidence.
"The next games are a month away yet and we need to keep everyone fit, but hopefully if we can get a strong squad together we've a chance."