The Gunners lost the round-of-32 clash on away goals as Youssef El Arabi scored in the final minute of extra-time to give the Greek side a 2-1 victory at the Emirates Stadium.
It was enough for Olympiakos to progress as the tie ended 2-2, with the away goals rule sending last season’s beaten finalists crashing out.
Pierre-Emerick Aubameyang’s acrobatic finish in the second half of extra-time appeared to have put Arsenal on course for qualification after Pape Abou Cisse’s header had sent the tie into an additional 30 minutes.
The early exit could end up costing the club tens of millions in lost revenue and throws serious doubt on whether they will be in European competition next season.
Arteta’s side sit ninth in the Premier League and face a tough end to the campaign, with fixtures against four of the current top-six still to play.
Arsenal had appeared to be turning a corner following Arteta’s appointment in December, winning three games in a row before this latest setback.
But now the Spaniard knows it is key for his squad to keep their collective heads up and not allow the entire season to end in disappointment.
“I’m the first one, I have to lift them, it’s my job,” he said.
“That’s the first thing I have to do. First of all we have to digest it because it’s very, very painful.
“I just want to concentrate now on lifting those players and getting back the belief, and convince them that there is still a lot to play for and move on.
“It’s part of this sport that a big disappointment can happen and it happened here. We have to learn and react as a team and as a club.”
Arsenal now face Portsmouth at Fratton Park in an FA Cup fifth-round clash on Monday night before returning to Premier League duties at home to struggling West Ham.
For some bizarre reason, the F365 Show still hasn’t been cancelled. So we’ll be back every Thursday with moreirreverent nonsense intriguing insight. Subscribe here.