The Black Cats lost 1-0 to their North-East rivals at the Stadium of Light as their cup run came to an end against Championship opposition.
Colback knows Sunderland have missed a golden opportunity to progress and admits their exit is particularly hard to take after they crashed out of last season's FA Cup quarter-final replay to Everton seven months ago.
The 22-year-old told the Sunderland Echo: "That wasn't good enough. We didn't create enough and we have missed a massive opportunity. It's a bitter pill to swallow.
"We weren't good enough all over the pitch. We made mistakes we shouldn't be making. We shouldn't be nervous because we've played the game long enough now and should be able to deal with that pressure.
"We wanted to do well in this competition. We missed a big chance last season in the FA Cup and this is another missed opportunity."
Colback knows Sunderland might have had a chance to do well in the tournament with the likes of Manchester City and Everton already eliminated, while either Chelsea or Manchester United will see their run ended on Wednesday night when they meet at Stamford Bridge.
"We want to play in finals and a few Premier League teams have dropped out of this cup," said the midfielder.
"It was a chance. We were at home and we expected to win the game. We let down ourselves and the fans."