The Hoops were made to pay for early profligacy less than two minutes into the second half when Rasmus Schuller fired the visitors ahead.
Parkhead striker Gary Hooper levelled in the 54th minute, before defender Charlie Mulgrew headed the winner seven minutes later.
Lennon will rue the moment of slackness which could ultimately end his Champions League dream, but the Parkhead boss will be heartened by the way his players reacted in adversity.
With a little bit of luck, Celtic, playing their first competitive game of the season, could have ended the game at least a couple of goals clear of a team who did not look any fitter or sharper -
despite being 19 games into their domestic season.
Lennon was boosted before the game when Kris Commons shrugged off an ankle knock, although striker Anthony Stokes, who also suffered an ankle injury in Saturday's 1-1 home friendly draw with Inter Milan, had to be content with a place on the substitutes' bench.
Skipper Scott Brown and winger James Forrest returned, with Victor Wanyama moving back from midfield to partner Mulgrew at centre-back.
The visitors had former Hearts striker Juho Makela leading the line.
Before the game, the home supporters looked up to the screens and cheered Hoops fan Michael Jamieson to his Olympic silver medal in the 200 metres breaststroke.
However, HJK Helsinki looked out of their depth at the start when Celtic could, and should, have raced into a two-goal lead.
Within 10 seconds Hooper came close with an angled shot after racing on to a Georgios Samaras head-flick, before Wanyama failed to connect properly at the back post from a Commons corner.
The Finnish side could hardly get out of their own half but in the 12th minute the home support held its breath when Wanyama barged into Makela inside the Celtic box near the bye-line but Czech Republic referee Miroslav Zelinka awarded a goal-kick.
Moments later in the visitors' box, it looked like Forrest had been nudged by defender Mikko Sumusalo before he missed with a header.
The Scotland winger then set up Hooper, who missed the ball altogether before Joe Ledley fired wide from 16 yards.
Wales international Ledley then forced a good save from goalkeeper Ville Wallen with a low drive from the edge of the box but again the home side could not capitalise on the corner, giving the Finnish outfit the encouragement to force their way into the game.
After slack play by Celtic left-back Emilio Izaguirre in losing possession, it took a great saving tackle by his opposite full-back Adam Matthews to deny Demba Savage getting his shot away.
Brown then went to ground in the Helsinki box following a challenge by midfielder Mika Vayrynen, but again the referee was unimpressed.
Antti Muurinen's men had lost all sense of dread by now and in the 35th minute Celtic goalkeeper Fraser Forster had to dive low to his right to push a long-distance drive from midfielder Joel Perovuo round the post.
The visitors' goal, two minutes after the restart, raised questions about the Hoops' defence.
The lively right midfielder Sebastian Mannstrom played in Schuller and when his first effort from 10 yards was blocked by Forster, he took advantage of a fortuitous rebound to squeeze the ball in from a tight angle.
Celtic Park was stunned, but Commons almost had the hosts level a minute later when his 25-yard drive beat Wallen but rebounded off the post and hit the goalkeeper on the head before flying clear.
However, the Scottish champions redressed the balance through Hooper after Samaras had picked up a Commons pass wide on the left and put a cross on a plate for the Englishman to tap into an empty net at the back post.
HJK Helsinki seemed to have lost their composure and just after the hour mark Mulgrew forced the Parkhead side into the lead when he met a Commons corner from the right and headed down and past Wallen from 10 yards.
Stokes then replaced Commons as Celtic continued to bombard the Finns' penalty area looking for the third goal.
In the 78th minute, with the home side camped inside the Helsinki half, Mulgrew sent a right-footed drive from 30 yards inches past the post.
With three minutes remaining, Hooper and Forrest ran into each other inside the HJK Helsinki box after a Stokes pass had opened up the defence and left the two of them clear.
It was an unusual moment of farce, but it was another missed opportunity missed which leaves Celtic's Champions League hopes on a knife-edge.