City were held to a 1-1 draw by Real Madrid at the Etihad Stadium and that result was enough to send Jose Mourinho's side and Borussia Dortmund into the last 16 with a game to spare.
But Hart admitted their troubles began a long time before Wednesday night, when City battled back from conceding an early goal to give Real a good game, creating a number of second-half chances that they could not convert.
Hart told Sky Sports: "It's very disappointing, we've had a few opportunities in this group where we could have done a lot better.
"It was tough tonight, obviously, and I don't think the referee's had a great game. I'm never one to blame them for things like that but I thought the ball was out of play for 20 minutes and I was very surprised how they played like that, but we're out of the competition."
Hart's only blemish was being beaten by Karim Benzema from close range, and he admitted City's defending left a lot to be desired.
"All goals are disappointing. A free man from seven yards out, that's sloppy," he admitted. "I don't know if it's a blessing but we're good enough to get back into games - we just didn't have enough time tonight. We move on to a big game on Sunday.
"All the games have cost us. We've lost our two away games and not won at home so I think you've got to look at both opportunities, we led in Amsterdam and Madrid so it's been a bad campaign for us.
"It is incredibly frustrating because we are a very good team and we've surprised ourselves in a very bad way. Unfortunately we weren't able to do the business when it mattered, and that's Champions League football."