Last summer's World Cup winners set up a gold medal match against the United States or Canada as they clung on desperately to win this afternoon's semi-final at Wembley, Elise Bussaglia missing a penalty that would have taken the game to extra-time.
Japan punished woeful set-piece defending to go 2-0 ahead, Sarah Bouhaddi's horrible howler gifting Yuki Ogimi the opening goal and non-existent marking allowing Mizuho Sakaguchi to double the lead.
France then went for broke, substitute Eugenie Le Sommer scoring with 14 minutes remaining and winning a penalty two minutes later, but they were ultimately left to fight it out for a bronze medal shortly before Thursday's final.
Japan, whose previous best Olympic performance was fourth place four years ago, came into today's game buoyed by Friday's quarter-final victory against Brazil.
They looked to get their renowned possession game going early on but suffered a seventh-minute scare when goalkeeper Miho Fukumoto horribly spilt a powder-puff shot from Louisa Necib, with no-one on hand to follow-up.
After Wendie Renard was booked for an ugly studs-up challenge on Shinobu Ohno, France keeper Bouhaddi produced an even worse error when she let Aya Miyama's 32nd-minute free-kick slip through her fingers.
Ogimi poked the loose ball goalward and although Sandrine Soubeyrand was on the line, the forward bundled it in.
Fukumoto got away with the first of several unconvincing punches as Japan maintained their lead, the world champions doubling it four minutes after the break thanks to another free-kick.
This time, Bouhaddi had her defence to blame as they went AWOL to allow Sakaguchi to glance another clipped Miyama free-kick into the far corner.
France threw on Le Sommer and Camille Abily, with Corine Franco, Marie-Laure Delie and Necib all wasting more than once chance to pull a goal back, although the latter could have done so but for an excellent low reaction save from Fukumoto.
The rampant French finally made their dominance pay six minutes later when Le Sommer smashed home Elodie Thomis' right-wing cutback.
Two minutes later, Sakaguchi tripped Le Sommer in the box and referee Quetzalli Alvarado pointed to the spot, but Bussaglia crumbled under the pressure, steering wide.
There was more drama to come, Delie unable to punish a poor punch and Ogimi hitting the post on the break before Fukumoto threw herself at a Renard shot in the fourth and final minute of stoppage-time.