City had to come from behind in Sunday's final at Wembley to eventually win 3-1 after Fabio Borini had fired the Black Cats into a 10th-minute lead.
Borini might have made it 2-0 but for a fine interception by Vincent Kompany before stunning strikes by Yaya Toure and Samir Nasri handed their side a lead which was cemented by substitute Jesus Navas' late effort.
Sunderland skipper O'Shea said: "Look, we gave it a good shot. Unfortunately, Mr Toure came up with the goal that he did, but for the first 55 minutes, it was going according to plan and a second goal would have been nice.
"But up until the last few minutes, we were in with a shout of bringing it into extra-time as well.
"Look, they scored two fantastic goals, don't get me wrong, but we gave them a good scare. They knew they were in a cup final."
O'Shea, 32, is currently on international duty with the Republic of Ireland, but will return to club action on Sunday when the Black Cats travel to Hull in the FA Cup quarter-finals with another trip to Wembley up for grabs.