Jake Hessenthaler swept in the 87th-minute winner from Rory Donnelly’s cushioned header after Gillingham’s outstanding playmaker Bradley Dack levelled in the 85th minute with a fine solo effort, coolly clipping the ball past Plymouth goalkeeper Luke McCormick after a surging run.
Argyle’s teenage midfielder Hiram Boateng had come close to breaking the deadlock in the 65th minute.
The 19-year-old from Crystal Palace sent a low 30-yard pile-driver just wide and then seconds later Curtis Nelson hooked a shot inches past the post as the tie opened up.
Gillingham countered, with Dack firing just wide two minutes later.
Plymouth introduced target man Ryan Brunt for winger Gregg Wylde in an effort to break the deadlock but it was first-half substitute Craig Tanner who volleyed the home side ahead from Kelvin Mellor’s cut-back in the 78th minute.
It looked like the home side would hold on for victory before the Gills’ late show.