But once again Hoops manager Neil Lennon was left to count the cost of a costly defensive mistake.
On the half-hour mark, with the game evenly-matched, a long punt by home keeper Benoit Costil was making its way harmlessly through to Fraser Forster but there appeared to be no communication between him and Cha Du-ri, who helped it on then watched in horror as the ball slipped past the Hoops keeper and in to the net.
That the South Korean was playing in midfield and only back as last man as a safeguard as Glenn Loovens and Charlie Mulgrew were still up the other end following a set-piece, made it harder to stomach for the Celtic boss.
It looked like another disappointing European night was on the cards for the Parkhead men but they mounted a spirited fightback and in the 70th minute Ledley flicked a Mulgrew free-kick over Costil to give the visitors a draw although they still remain bottom of Group I on goal difference.
Once again Lennon changed his defence, this time dropping stopper Daniel Majstorovic and replacing him with Loovens.
The Sweden international paid the price for the Hoops' woeful first-half display at Kilmarnock on Saturday, where they went in at the interval 3-0 down before fighting back to salvage a point.
Midfielder Victor Wanyama came in for the injured striker Gary Hooper, who did not travel, leaving Anthony Stokes to lead the line.
As promised, Rennes coach Frederic Antonetti left out his star midfielder Yann M'Vila altogether with other regulars such as Jonathan Pitroipa, Victor Montano and Jires Kembo-Ekoko starting on the bench.
Celtic started confidently enough and midfielder Ki Sung-yueng had an early drive from long range which forced a save from Costil.
Ki then went down in the box following a challenge from Vincent Pajot but his claims for a penalty went unheeded by Polish referee Robert Malek.
The French club gradually worked their way in to the game and in the 12th minute skipper Kader Mangane headed a looping free-kick from midfielder Julien Feret wide of the target at the back post, before a drive by Youssouf Hadji was headed over by Wanyama for a corner, which the visitors survived.
Rennes had control of the match as it passed the halfway stage but Forster had yet to see any real action.
The Parkhead side's first real chance came in the 24th minute when stand-in skipper Mulgrew threaded a long pass through to Stokes who smartly turned Mangane to leave him for dead but Costil turned the Irishman's shot over the bar for a corner which came to nothing.
That near miss appeared to energise Lennon's side but they were undone on the half-hour mark when the mix up between Du-ri and Forster handed the home side the lead.
It was a sickener for Celtic who almost went two behind moments later but Hadji's header close-range header from Alexander Tettey's cross went over the bar.
Celtic still looked a bit shell-shocked at the start of the second half, following what had no doubt been a robust pep-talk by Lennon.
In the 49th minute, though, Loovens did well to deny Tettey who had burst in to the box to get on to a pass by Abdoulrazak Boukari, the Dutchman shepherding the ball back to Forster.
Stokes continued to plough a lone furrow in attack and while Rennes were hardly piling on the pressure, it was hard to see where Celtic's equaliser was coming from.
In the 55th minute Forster, who was being taunted by the home fans, redeemed himself in part when he made a great point-blank save from defender Chris Mavinga who had worked a one-two with Feret before bursting in to the Celtic box.
Rennes then brought on Pitroipa and Kembo-Ekoko for Yacine Brahimi and Boukari but in the 63rd minute Costil made another save from Stokes, who had fired rather too straight at the keeper.
Moments later, as Celtic rallied, Costil parried a shot from Cha to the feet of Ki but slammed the ball over the bar from the edge of the box.
The game swung from end to end and in the 70th minute Celtic got what they deserved when midfielder Joe Ledley jumped high in a packed Rennes box to head in a Mulgrew free-kick.
The match hurtled towards an exciting finale and five minutes later, after Stokes had been fouled by Mangane, Mulgrew fired the free-kick from 30 yards just past the post.
In the 79th minute Forster made another great point-blank save, this time from Hadji's over-head kick.
Both sides went all out for the winner and in the dying seconds Forster made yet another great block from Pitroipa but the Hoops were worthy of a point which Lennon and the away fans celebrated with some gusto at the final whistle.