The newly-promoted side were full value for their victory as they came from behind to take all three points with Aron Gunnarsson's strike and Campbell's late brace cancelling out Edin Dzeko's opener.
Substitute Alvaro Negredo pulled one back deep into stoppage time, but Cardiff held on to secure their first win back in England's top flight.
Cardiff started brightly and had claims for a penalty waved away inside the opening minute when Campbell went down in the box under pressure from Fernandinho, but referee Lee Probert ignored the home side's claims.
The home side continued to take the game to Manchester City early on and, on six minutes, Peter Whittingham curled a free-kick just wide of Joe Hart's left-hand post after Craig Bellamy had been fouled.
Cardiff were showing great energy to close down Manchester City every time they got on the ball and the visitors had to wait until ten minutes before the break to produce their first shot on target when David Silva's curling effort was easily saved by David Marshall.
Campbell had two chances to break the deadlock in the closing minutes when he got in on goal behind some hesitant Manchester City defending.
For the first chance Gunnarsson sent Campbell clear of Javi Garcia and Joleon Lescott, but the striker's touch let him down and on the stretch he could only poke his shot wide of Hart's goal.
A minute later Garcia failed to deal with Marshall's long punt upfield, heading it right into the path of Campbell, but Hart was quickly off his line to smother his effort.
Somewhat against the run of play Manchester City broke the deadlock on 52 minutes with a super strike from Dzeko. Sergio Aguero laid the ball off to Dzeko 25 yards from goal and the Bosnian drilled in an unstoppable right-footed shot which took a slight deflection before flying past Marshall.
Cardiff responded superbly to going behind and they deservedly drew level on the hour mark with Gunnarsson scoring their first Premier League goal.
Kim Bo-Kyung worked his way down the right with a fine run and his cross found Campbell at the near post, who saw his close-range effort brilliantly saved by the legs of Hart, but Gunnarsson was on hand to smash home the rebound and send the home fans into raptures.
Manchester City started to take control of the game in the final 15 minutes and Yaya Toure and Silva both curled efforts from the edge of the box over the top.
However, Cardiff grabbed the lead 11 minutes from time with Hart failing to deal with Whittingham's fierce in-swinging corner and Campbell was on hand to head home from close range.
The home side went 3-1 up three minutes from time as Manchester City again failed to deal with a set-piece as Campbell rose highest to head home substitute Don Cowie's corner to spark wild celebrations in the stands.
Negredo set up a tense finale when he headed home his first goal for Manchester City in injury time and Aguero had the final chance when he fired just wide in the closing seconds as Cardiff held on for a memorable victory.