Chris McCann gave the Championship outfit an 18th minute lead but Fraizer Campbell equalised for the hosts nine minutes later.
Ben Watson - the man who scored the only goal in last season's final victory over Manchester City - then won the game at the Cardiff City Stadium with a terrific strike five minutes before the break.
A delighted Rosler, whose side saw off Crystal Palace by the same scoreline in the last round, said: "We take it game by game. I felt very happy with the performance, the application.
"I thought our supporters, they travelled in big numbers. We made them proud, we wanted to make them proud.
"I wanted to compete against the best teams in the country and Cardiff today gave us the possibility, and we stood our own ground very well.
"We've beaten now two teams (Premier League) on the way and we're now in the last eight."
He admitted he did not witness the sparkling goal Watson scored.
"I didn't see the second goal. I just turned around and the ball was in."
Bluebirds boss Ole Gunnar Solskjaer now has to focus on a Premier League relegation battle and Rosler offered his counterpart his support.
He said: "We are friends and Ole Gunnar is a winner. I said that before the game.
"He knew exactly what he got himself into coming to Cardiff and he can make it happen. He knows exactly what he's doing and you see already what he tried to achieve in terms of styles of play.
"He wants to attack and he will be fine. I have no worries about that."