The Greek side had just two shots on target all night, but after skipper Vassilas Torosidis scored his first goal in over two years, substitute Mitroglou tapped home the winner in the first minute of injury time.
It was rough justice on Montpellier, who had deservedly led through Gaetan Charbonnier's clinical finish just after half-time.
But their failure to convert a host of chances thereafter was cruelly punished to leave them bottom of Group B with just a single point.
Olympiacos had lost their opening two matches and looked second best for a large part of the opening period.
Montpellier's rough diamond striker Younes Belhanda fizzed a low shot across goal from the edge of the area before the hosts thought they had a penalty on 11 minutes.
Kostas Manolas handled in the area but the referee called the play back for an earlier foul just outside the area.
Montpellier boss Rene Girard, who was serving a touchline ban, cut a frustrated figure in the stands and his mood would hardly have been helped when Anthony Mounier sliced his shot from the ensuing set-piece high and wide.
Belhanda blazed over from 30 yards after strong lead-up work by Mounier but the hosts' early pressure began to fizzle out as the Greek side settled.
Benjamin Stambouli had to block a Rafik Djebbour header at close quarters before the striker just failed to get a foot on Leandro Greco's dangerous cross from the left.
Greco caused more problems down his flank but Paulo Machado wasted the chance.
As the half ticked down, Montpellier returned fire and Charbonnier almost dinked a shot over Roy Carroll but the goalkeeper was alive to the danger.
The hosts did take the lead just three minutes after half-time though thanks to some quick thinking from Charbonnier.
The striker initially flicked a header forward, and when the defence was only able to poke the ball back into his path, he lashed his shot in from just inside the area.
Olympiacos passed up a glorious chance to draw level moments later when Pablo Contreras floated a cross to an unmarked David Fuster at the back post but he headed wide.
Montpellier then should have killed off the game but passed up a series of chances.
John Utaka forced Carroll into a save before Belhanda missed their best opening. The striker latched on to another weak clearance but after feigning by a defender he arced his shot wide.
It proved costly as Olympiacos drew level on 73 minutes after Giannis Maniatis won a free-kick 35 yards from goal.
Jose Holebas whipped the set-piece to the top of the six-yard box where skipper Torosidis was able to take a touch, after his marker Benjamin Stambouli slipped, before volleying home.
Montpellier pressed forward in search of a winner they just about deserved, but were instead caught on the counter-attack.
After losing the ball on halfway the French side still had a chance to clear the threat, but Henri Bedimo was caught on the ball and Djamel Abdoun raced clear and centred for Mitroglou to tap in.