Substitute Nicklas Bendtner scored in the 88th minute, hammering in from close range after David Marshall produced an amazing save from Bacary Sagna's header.
Then, as Bendtner limped off after picking up an injury in the process of scoring, Theo Walcott ran onto Jack Wilshere's neat flick to lift the ball over the advancing keeper to seal the three points.
Defeat was hard on Cardiff who turned in a battling performance in front of manager-in-waiting Ole Gunnar Solskjaer.
The former Manchester United striker sat alongside Bluebirds owner Vincent Tan to watch the visitors frustrate Arsenal for long periods.
Lacking the injured Mesut Ozil, Olivier Giroud and Aaron Ramsey, the Gunners struggled to find a way past the massed ranks of the visitors until their late rally.
The closest they came was a Wilshere shot which cannoned off the upright while they could have been left to rue two golden chances for Per Mertesacker who twice headed wide from just six yards.
Wilshere also had a strong appeal for a penalty rejected by referee John Moss after going down under a challenge from Gary Medel while, at the other end, Cardiff shouted for a spot-kick of their own as the ball appeared to strike Nacho Monreal's arm.
Solskjaer travelled to London on New Year's Day as talks stepped up over him replacing the sacked Malky Mackay, and the appointment could be confirmed on Thursday if negotiations between the Molde boss and Cardiff owner Tan progress well.
But Solskjaer will have his work cut out to keep his new club clear of the relegation battle following just one win in the last 10 Premier League games.
Arsenal, meanwhile, again answered their critics to dig deep and overcome their injury problems to leapfrog Manchester City back to the summit.
It was Walcott who provided the main outlet on a slick pitch as Arsenal aimed to play him away behind the Cardiff defence.
On 14 minutes, Wilshere whipped a low ball in from the left which Walcott was just unable to reach as he slid in at the far post and put his shot into the side-netting.
Arsenal slowly started to dominate possession, with Santi Cazorla firing wide from 20 yards after another positive break down the right.
The home fans in the Clock End were screaming for a penalty on 25 minutes when Medel looked to have tripped Wilshere and in a rare Cardiff attack, Jordon Mutch got away down the left and drilled in a low shot which Wojciech Szczesny kicked clear.
Clever play by Cazorla in the left side of Cardiff box saw the Spaniard pick out a ball back to Walcott, but his snapshot flew wide.
Cardiff felt they should have been given a penalty when a cross from Craig Noone came off Monreal's boot and up on to his arm, but the officials were having none of it.
Arsenal continued to press, but lacked a decisive pass in the final third. Wilshere collected the ball on the left before cutting back into the Cardiff penalty area and his fierce angled drive cannoned back off the outside of the near post.
On 65 minutes, Gunners boss Arsene Wenger made a double change as Tomas Rosicky and Bendtner replaced Mathieu Flamini and Lucas Podolski.
It remained one-way traffic heading into the final 15 minutes but the visitors' resistance was finally broken on 88 minutes. Monreal floated over a deep cross from the left, which Sagna headed goalwards and after Marshall made a fine reaction block, Bendtner was alert to stab the loose ball home.
Then in stoppage time, Walcott was sent clear by Wilshere and clipped the ball over Marshall to seal another hard-earned victory.