Roberto Mancini's battling men seemed to have salvaged a point having trailed at half-time to a Wayne Rooney brace, only for Robin van Persie to stroke in a deflected free-kick winner in the dying seconds.
Securing all the spoils to stretch their lead at the top of the Premier League would have been pleasing for United.
But it will have also helped to reduce some of the pain of losing twice to their rivals last season, including the 6-1 demolition at Old Trafford.
"It was a fantastic game and you couldn't take your eyes off it," Ferguson said.
"They hadn't lost in two years at home and both of us are contending at the top of the league. We should have finished it off though."
United were unlucky not to have scored a third, with Ashley Young incorrectly ruled offside when he tapped in after Van Persie had curled a shot against the upright.
Yaya Toure and Pablo Zabaleta were on target to level the scores, before the late drama and the Dutchman's last-gasp strike.
"Instead of it being 3-0 - and it was a perfectly onside goal - they go and score. Up to then we were far better," Ferguson added.
"That's the kind of game it is - it can kick you in the teeth. They deserved to get back in it - they fought like hell and gambled with three strikers.
"We can be proud of the fact that, opposed to last year when we sat off them too much, gave them far too much respect, there was a great team spirit.
"They were determined to win the match. I think in general everyone would recognise that."