Caley Thistle's top scorer Adam Rooney opened the scoring from the penalty spot after 44 minutes, but Ledley levelled from close range two minutes into first-half stoppage time at the Caledonian Stadium.
The midfielder netted the decider from three yards after 68 minutes to secure a slot in the last four for the SPL leaders.
Ledley was one of four changes to the Hoops side which beat Hamilton 2-0 last time out.
Star striker Gary Hooper returned to the Celtic's starting line-up after recovering from a knee injury, while Mexican Efrain Juarez and Dutchman Glenn Loovens were also back for Neil Lennon's men.
Top scorer Anthony Stokes was dropped to the bench, while Scott Brown and Mark Wilson were suspended and Daniel Majstorovic injured.
Meanwhile, Caley Thistle were unchanged from the side which beat Motherwell 3-0 on their last outing, but attackers Jonny Hayes and Aaron Doran were fit enough to be among the substitutes.
Despite it being all one-way traffic in the early stage, SPL leaders Celtic failed to score.
Celtic started brightly with Loovens heading an inswinging Kris Commons corner inches over after just three minutes.
Inverness goalkeeper Ryan Esson was called into action in 12 minutes when he saved a cross-come-shot from Emilio Izaguirre.
The Honduran full-back was the Hoops' most threatening player in the opening stages, terrorising the Caley Thistle defence with his power and speed.
Commons was next to try his luck when he fired wide from 25 yards.
Caley Thistle captain Richie Foran squandered the hosts' first chance midway through the half. The Irishman blasted across the face of goal from the edge of the six-yard box.
His angled volley at full stretch was set-up by a deep cross from Shane Sutherland.
Celtic responded with their captain Georgios Samaras powering a header wide. The Greek attacker got in ahead of defender Chris Hogg to connect with Charlie Mulgrew's cross.
Inverness broke the deadlock a minute before the interval when Rooney netted from the penalty spot.
The Irish striker despatched the spot-kick after Foran had been felled in the box by Loovens after being played through by Russell Duncan.
Ledley levelled from six yards in the second minute of stoppage time when he connected with Izaguirre's cross.
Ledley might have added a second five minutes after the restart when his header forced a save from Esson. The midfielder nodded a Commons corner towards goal but Esson was equal to it.
Shane Sutherland might have regained Inverness the lead after 56 minutes if not for a diving save from Fraser Forster.
The 20-year-old sent in an angled thunderbolt from just inside the box after a header down from Foran.
Just after the hour the visitors were back in control and Commons sent a 30-yard free-kick just wide.
Ledley put Celtic in front after 68 minutes when he headed home from close range.
The Welshman was found by a free-kick from Ki Sung-yueng after Stuart Duff fouled Izaguirre to collect the first booking of the game.
Moments later Nick Ross might have equalised when Foran drilled in a cross but the teenager's effort was deflected wide.
Esson kept Caley Thistle in contention with five minutes to go when he saved a rasping Hooper shot from outside the area.
However Inverness failed to find the cutting edge in the closing stages and Celtic advanced.