There was nothing between the sides over the 90 minutes with Simon Clist cancelling out Craig Conway's early opener.
However, Malky Mackay's side's class shone as the game wore on and Whittingham settled the tie in the 98th minute before Nathaniel Jarvis added further gloss to the scoreline.
Mackay opted to make several changes to his starting line-up, with Kenny Miller - who scored the solitary goal in Cardiff's season-opening victory over West Ham on Sunday - and Robert Earnshaw not included in the squad.
Despite playing two divisions below their visitors, it was the Us who enjoyed the better of the opening exchanges.
However, they were struggling to fashion opportunities and were given a warning as Andrew Taylor saw his fizzing effort sail just wide.
Clearly not learning their lesson, Oxford fell behind in the 12th minute, albeit in unfortunate circumstances.
Having been found by Jon Parkin's cushioned header, Conway found himself in space and unleashed a fierce strike which deflected off Jake Wright and into the net.
The hosts responded well, though, and continued to enjoy an impressive share of possession.
Indeed, their industrious approach was rewarded in the 30th minute as Clist levelled matters.
Jon-Paul Pittman, having just been harshly denied a penalty, raced clear and saw his effort parried into the path of Clist, who made no mistake.
Clearly spurred on by their equaliser before half-time, the hosts continued to press forward after the interval as Asa Hall struck just wide.
But as the final whistle drew closer, Cardiff became increasingly dominant and thought they had won it when the ball appeared to cross the line after a goalmouth scramble.
However, the officials deemed it not a goal to herald extra time.
The break in proceedings did not halt Mackay's troops' rush though, and Whittingham found the net with a low drive from just outside the area.
Jarvis then added insult to injury by capitalising on a defensive mix-up to head home in the dying seconds.