The shock champions of eight years ago turned Group A upside down in a remarkable final round of fixtures in Poland on Saturday evening.
Karagounis' winner - on the night he equalled a national record by earning his 120th cap - came completely against the run of play on the stroke of half-time at Warsaw's National Stadium.
But no-one could argue Greece did not deserve their victory at the final whistle, with 2004 hero Karagounis denied a blatant penalty - and picking up a booking that ruled him out of the next match.
Russia, who had thrashed Czech Republic 4-1 in their opening game, barely tested goalkeeper Michalis Sifakis and were made to pay as the Czechs beat Poland in Wroclaw to steal top spot.
Dick Advocaat's men went into the game in the dangerous position of knowing they needed only a draw to reach the quarter-finals, with even a defeat not necessarily fatal.
It was win or bust for their opponents, who got off to the fast start they failed to produce in their previous games by winning a succession of early corners.
Vyacheslav Malafeev made an acrobatic save from one of them when Kostas Katsouranis hooked Karagounis' sixth-minute delivery within reach of the goalkeeper - although it was going wide anyway.
Vassilis Torossidis also had a shot blocked before Russia's technical superiority began to show, Andrey Arshavin poking Alan Dzagoev's 10th-minute cross straight at Sifakis and Aleksandr Kerzhakov inches wide with a superb 20-yard half-volley.
Yuri Zhirkov spoilt a thrilling break with a wayward cutback and Kerzhakov should have done better seven minutes from half-time when he produced an embarrassing air shot from Dzagoev's knockdown.
Zhirkov also sent a stunning long-ranger inches over the top as the half looked to end goalless until Karagounis pounced on the former Chelsea winger's loose header before drilling the ball under Malafeev.
Russia boss Advocaat took immediate action by hauling off Kerzhakov for Roman Pavlyuchenko after the interval before seeing Roman Shirokov and Igor Denisov both fail from long range.
Greece would have doubled their lead shortly before the hour mark had Aleksandr Anyukov not succeeded with a desperate clearance to prevent Theofanis Gekas turning in Torossidis' cross.
And they were denied a 61st-minute penalty when Karagounis danced his way through the Russia defence and went down after touching legs with Sergei Ignashevich.
There was definite contact but referee Jonas Eriksson thought otherwise and booked a distraught Karagounis.
Manager Fernando Santos quickly withdrew the midfielder for Grigoris Makos after sending on Jose Holebas for Gekas.
Zhirkov was yellow carded for hauling back Dimitris Salpingidis and was lucky to avoid further punishment when Giorgos Tzavellas curled a beautiful free-kick against the angle of post and crossbar.
Russia were losing their discipline and Tzavellas was the victim of a crude challenge by Dzagoev, who was booked.
Advocaat threw on Pavel Pogrebnyak for Denis Glushakov as news the Czechs had taken the lead in their game - leaving his side needing to score - filtered through.
Denisov drilled a fine first-time strike too close to Sifakis before Russia's last throw of the dice saw Marat Izmailov replace Anyukov.
Greece responded by withdrawing Salpingidis for Sotiris Ninis for the final seven minutes as the men in red poured forward, Dzagoev glancing Arshavin's header inches beyond the post.
It was all Russia in the dying seconds but all they could manage was a Pogrebnyak booking for deliberate handball.