United thrillingly led 3-0 after a remarkable first half in which Rooney headed two goals either side of a Danny Welbeck effort, and City captain Vincent Kompany was sent off.
City hit back after the break with goals through Aleksandar Kolarov and Sergio Aguero, but United, for whom Paul Scholes came out of retirement to make a substitute appearance, held on to end City's cup defence in the third round.
The game, played on a greasy surface in continual drizzle, more than lived up to all the pre-match hype.
At one stage, it seemed United were on course to avenge their 6-1 thrashing last October in the most incredible fashion but City's recovery left them to settle for a narrow win.
Nevertheless, it was Premier League leaders City's first home defeat since December 2010 and reminded them that United remain in the way of their bid for domestic dominance.
The afternoon began unbelievably enough as United announced prior to kick-off that 37-year-old Scholes had rejoined the playing staff to help them through their injury problems and would be on the bench, having retired at the end of last season.
Yet what followed was even more dramatic as Rooney, after speculation over his own future, made an emphatic statement and United, initially, were ruthless in taking advantage of Kompany's dismissal.
There was little hint of that as City pressed United's reorganised defence in the opening minutes.
Chris Smalling slotted in at centre-back alongside Rio Ferdinand while Phil Jones was moved to right-back.
Anders Lindegaard, again preferred to David de Gea in goal, was almost caught out by Sergio Aguero as he delayed a clearance but escaped with a goal kick.
The game exploded into life as United took the lead against the run of play in the 10th minute.
Antonio Valencia found space on the right as the visitors counter-attacked and delivered a cross for Rooney, who had laid the ball off to the winger, to power home a superb header off the underside of the bar.
The England striker showed great technique as he rose above the City defence and then ran away to celebrate clutching the United badge on his shirt.
City were then dealt an immediate blow to their hopes of making a quick replay as Kompany was sent off for a challenge on Nani.
Kompany clearly won the ball but referee Chris Foy showed a straight red card for the offence of lunging forward with both feet.
City were angry with the decision but recovered their composure and Aguero brought a fine save from Lindegaard from distance.
David Silva then had a shot deflected wide and Aguero fired into the side-netting but United remained a threat on the break and took control after 30 minutes.
Nani was denied shooting space on the edge of the box but he fed the ball wide to Patrice Evra, who crossed low.
Samir Nasri attempted to clear but did not make firm contact and Welbeck hooked in his seventh goal of the season on the volley.
Things got worse for City as they conceded a penalty six minutes before half-time when Kolarov brought down Welbeck.
Costel Pantilimon, playing in place of the rested Joe Hart, saved Rooney's spot-kick but was unable to deny the United forward as he followed up to head in the rebound.
It seemed the best City could achieve in the second half was damage limitation, but they gave themselves hope as Kolarov pulled one back after 48 minutes.
Evra was booked for felling Micah Richards outside the area and Kolarov stepped up to bend a superb free-kick around the wall and into the net.
Scholes came off the bench to make an unexpected 677th United appearance when he replaced Nani just before the hour.
United wanted another penalty moments later when Valencia went over as Kolarov challenged him just inside the box but Foy gave nothing.
City got themselves right back into the game after 65 minutes as Milner nicked the ball off Scholes following a throw-in and whipped the ball in for Aguero.
The Argentinian shot straight at Lindegaard but the Dane could only parry and Aguero blasted in the loose ball.
Scholes got his first sight of goal when he struck a trademark long-range shot after 71 minutes but Pantilimon saved comfortably.
City sent on former United midfielder Owen Hargreaves in place of Nasri with eight minutes remaining.
City appealed long, but in vain, for a penalty when a low ball from Kolarov took a ricochet and appeared to catch Jones' arm.
And City had United holding on as they pressed at every opportunity, but the ball would not fall for them in the box.
Richards attempted a spectacular overhead kick but failed to make contact.
Kolarov tested Lindegaard with a free-kick in injury time but United, with victory in sight, scrambled the ball clear.