Wales led in the first period through a long-range Jason Koumas strike, and should have been further ahead at the break.
But Georgia's desire for an emotional victory, given the nature of their war-torn country, was greater than Wales', and they equalised through Levan Kenia and then grabbed an injury-time winner through substitute Beka Gotsiridze.
It was no more than they deserved. Wales had fallen away badly after such a bright, positive opening at the Liberty Stadium.
Georgia overcame considerable difficulties to be in Swansea for this friendly due to the war with Russia, and a hastily patched-together squad was captained by Blackburn's Zurab Khizanishvili and included a host of home-based players.
For Wales, the night included a parade of the seven surviving members of the Wales squad that played in the World Cup of 1958 - Ken Jones, Cliff Jones, Terry Medwin, Mel Hopkins, Stuart Williams, Mel Charles and Colin Baker.
No Wales side in the 50 years since has ever reached a tournament finals.
And following the crushing blow of the Under-21s' defeat in their crucial UEFA qualifier in Wrexham to Romania earlier in the evening, there was a realisation those heroes of the past may not be emulated any time soon.
The Georgians wore black armbands and linked hands, held aloft, when their anthem was played, Georgian flags waved from a small number of fans who had made their way to Swansea for what was understandably an emotional occasion.
Georgia may well have had problems getting their top men here, but those that did figure showed class, passing ability and excellent movement.
Khizanishvili and Levan Mchedlidze both saw headers go close in the opening minutes.
But Wales slowly worked their way into the game, with Simon Davies firing over from five yards after Parry's fine cross.
Wales then took the lead after 16 minutes when Koumas cracked in a vicious 25-yarder that goalkeeper Giorgi Loria half-stopped, but allowed the ball to bounce out of his grasp and over the line.
Loria was forced into a plunging save to stop a Parry drive, and the Cardiff man went close again from a Davies pulled-back cross.
Koumas was being allowed the time to run the show and he was finding space at will, with the Georgians failing to put him under much pressure.
Neil Eardley and Davies combined well down the right after 26 minutes to produce a delivery that evaded Loria and saw Koumas fire into the side netting.
Koumas, Parry and Davies were working well together, while Freddy Eastwood looked lively. One run past three men by the Coventry man ended with Loria forced into another fortunate block.
Georgia started the second period with a quicker tempo and generally more urgency.
Substitute Rati Aleksidze saw a header go just wide, with Levan Kenia firing over soon after.
Wales were pressured on both flanks, with plenty of good delivery.
And it was no surprise when Georgia equalised after 66 minutes. Aleksidze's ball cut through the centre of the Wales defence to allow Kenia the opportunity to guide the ball past Boaz Myhill.
There was a hint of offside about the goal, but it was clinically taken.
Wales hit back when Davies was played in by Koumas, and the Fulham man got round Loria to produce a shot that Levan Khamaladze cleared from in front of an open goal.
Parry had drifted from the game by now after a promising hour, and David Vaughan replaced him after 70 minutes.
Robert Earnshaw finally got back into the fray when he replaced Eastwood after 79 minutes, having been left out of the last four squads.
But Wales were struggling, generally, at this stage. Georgia having looked the better side in the second-half.
Koumas' impact had been nothing like as impressive as his first-half contribution. He had switched from wing to wing without much effect.
And Georgia punished Wales with an injury-time winner when Beka Gotsiridze took advantage of a mix-up between Ashley Williams and Myhill to head home.