An early strike from Fabio Borini had given the Black Cats a shock lead, but quick-fire goals from Yaya Toure and Samir Nasri turned the game on its head before Jesus Navas wrapped things up with a late third.
The Wearsiders put in a valiant display as they frustrated Manuel Pellegrini's side for large parts of the tie, and Poyet was full of praise for his men after coming into the game as underdogs.
"I think that's the main thing to talk about, the way we played against one of the top teams in the world with plenty of power, quality, money, the best players," he told Sky Sports.
"We had a go, we were 1-0 up, we had the chance to score the second goal and maybe that was the difference today.
"We knew that everything had to go for us, we knew it. Our little foul was not given, a wonder goal from Yaya Toure and then an incredible finish from Nasri.
"There was things that you expect today not to happen. But I'm proud of the players, I'm proud of the fans. I can promise them that we gave the best shot that we had."
It took City until the 55th minute to draw level with Sunderland and Poyet says that the supporters who travelled down to Wembley in their masses can leave with their heads held high.
"I think for 50 plus minutes, we were more than a decent team and I think the fans need to stay proud because the players tried their best," he said.
Asked how to prevent City's wonder goals, Poyet added: "I think the only way is to put the goalkeeper in that place before he shoots, you see it coming but he cannot be there.
"For the second one, I don't know if Nasri hit it with a full boot or a bit of shin as well but it was an absolutely outstanding finish.
"That's the best part of what we are all about. To beat us today, Manchester City had to score two wonder goals and on another day if they don't, they would have a problem."
Sunderland remain in a battle to secure their Premier League status but they could still return to Wembley this season if they can overcome Hull City in their FA Cup quarter-final.
Poyet is now hoping to use the cup final performance as a platform for the rest of the season and inspiration to reach the FA Cup semi-finals.
"It's difficult to take. I hate losing but I'm proud of the players," he said.
"We're talking about the rest of them (games) now. Now it's up to us. I think we should do this every week.
"If we do this every week, we'll stay up in the Premier League and we'll have a great chance to come back."