A dramatic stoppage-time goal from John Sutton gave the hosts a point after Celtic had fought back brilliantly from 2-0 down to go 3-2 up.
Goals from Sutton and Zaine Francis-Angol either side of a Gunnar Nielsen penalty save from Kris Commons had put the home side 2-0 ahead before Anthony Stokes reduced the deficit just before the interval with a superb chip.
Georgios Samaras levelled 10 minutes after the break before Leigh Griffiths, on for Beram Kayal, grabbed a third in the 84th minute but, as the game moved in to stoppage-time, Sutton grabbed his second from a yard out to take them to within three points of second-placed Aberdeen.
Lennon, who sent his players out early for the start of the second-half after giving them a "bit of a rocket", said: "I am annoyed. We were poor defensively first half, gifted Motherwell two goals with poor defending.
"There was definitely a psychological edge missing first half. We got all that back in the second half, we were fantastic, and we should have won the game.
"We dominated the second-half from start to finish but switched off. Motherwell had one attack and scored from it.
"On the counter attack we had three four or five really good chances to make the game safe.
"The keeper has made a great save from the penalty, so overall, attacking wise, we were really very good, defensively we were sloppy.
"We were done with a straight ball for the first one but I think the keeper should save it, the second one we have missed the flight of a cross and the third one Emilio (Izaguirre) is sleeping and they have got in behind us.
"But I am really pleased with the quality and character in the second half."
Stokes insisted he had meant to chip Nielsen from inside the box for his goal.
"I went for goal but it's a one in a thousand. I didn't have much on and couldn't cut it back so I just went for it. I wouldn't say I was lucky but I'll take it as a good finish," he said.
"The half-time team talk was quite short and had we not scored just before half time it could've been even shorter."
Motherwell boss Stuart McCall said: "I really enjoyed that, I'm so proud of the lads today. I thought we played our best football at 2-2.
"We've done it so often this season, we lose silly goals right on half-time but it was a great goal by Stokes. We knew Celtic would come at us in the second half and they got an early goal.
"But we got what we deserved. There were some massive performances from the players.
"They looked dead on their feet but they dug out a point and next week's game against Dundee United is massive."