Arsenal had already secured top spot in the group before the game in Athens but Gunners boss Arsene Wenger - taking charge of his 200th European tie - will still have expected better.
In-form Algeria striker Rafik Djebbour slotted home on 16 minutes after a loose ball was not cut out at the edge of the area, before substitute goalkeeper Vito Mannone - on for the injured Lukasz Fabianski - made a hash of his clearance as David Fuster made it 2-0 on 36 minutes.
Yossi Benayoun scored a terrific volley to reduce the deficit on 57 minutes before Francois Modesto secured victory with a minute left.
However, with Marseille producing a late rally to win at Borussia Dortmund, it was not enough to see the Greeks through to the last 16.
The Georgios Karaiskakis Stadium was rocking as the home side started on the offensive, with Arsenal barely able to get out of their own half during the opening five minutes.
Suddenly the visitors burst clean though when Marouane Chamakh was found in space down the right. His cut-back picked out the on-rushing Andrey Arshavin, who darted into the Olympiacos penalty area, only to shoot straight at Hungarian keeper Balazs Megyeri.
Olympiacos, though, were soon back on the attack as lively Belgian winger Kevin Mirallas whipped a low ball across the Arsenal six-yard box from the left.
At the other end, Andre Santos' centre only needed a touch from Chamakh, but the Moroccan just could not get there.
Emmanuel Frimpong then shaved the woodwork after more pressing by Chamakh.
Olympiacos had the lead on 16 minutes when Arshavin failed to break after a quick throw from Fabianski.
The ball was worked back up to Fuster, with neither Johan Djourou nor Sebastien Squillaci able to cut out the Spaniard's first-time pass. That released Djebbour into the left side of the Arsenal penalty box, where he rounded the keeper to slot in from a tight angle.
The Gunners' back line was in disarray as first a poor backpass from captain Thomas Vermaelen saw Mirallas almost send the ball back over the keeper's head, and then another low cross from the left caused havoc in the Arsenal six-yard box.
Fabianski injured his knee in the melee, and had to be carried off on a stretcher after 25 minutes, replaced by 23-year-old Italian Mannone - whose last appearance was on loan at Hull in April.
Alex Oxlade-Chamberlain did well to get to the goal line and cross towards the back post, where Benayoun hooked his shot wide.
Mannone's blunder handed Olympiacos a second on 36 minutes.
The Italian had initially done well to race out to the edge of his area to head clear under pressure from Mirallas - only to then make a complete hash of attempting to scissor-kick the ball clear, thinking he was not yet back in the box, when it was returned over him by Fuster and bounced into the net.
The half-time whistle was a welcome relief for the visitors - not that much could be heard above the cacophony of a vocal home support.
Mannone made a good reaction save at the start of the second half, getting down quickly to block Modesto's far post shot, before former Aston Villa defender Olof Mellberg clipped the top of the angle after getting a free header from a corner.
On 51 minutes, Arsenal were forced into another change when Santos had to be helped off with what looked like a leg injury, replaced by 19-year-old Spaniard Ignasi Miquel.
Arsenal pulled a goal back on 57 minutes when Miquel's floated cross was chested down by Chamakh and Benayoun lashed the ball into the far corner.
Mannone then almost hit the self-destruct button again as he failed to collect a loose ball when Vermaelen stood off - and somehow Modesto headed wide of an open goal from six yards.
Benayoun's flick release Tomas Rosicky into the Olympiacos penalty area, but Megyeri was out quickly to block.
Mannone produced another good reaction save to deny Mirallas from close range, before Modesto slotted home with a minute left after Mellberg's header hit the post.
However, with Marseille producing a late rally to win 3-2 in Germany, it was not enough for the Greek side to advance to the knockout phase.