The Pilgrims got off to the worst possible start at the Avenue Stadium when Conor Hourihane picked up two yellow cards inside the opening 10 minutes and were eventually punished after the break.
Exeter loanee Gosling, released by Argyle aged 16, smashed home the winner from close range to book the Magpies a second-round clash at fellow non-League outfits Luton or Nuneaton.
Despite their numerical disadvantage, Argyle still managed to create the first genuine chance of the game when Onismor Bhasera hit the side-netting in the 25th minute.
Two minutes later, Charlie Clough hit the upright with a headed effort for the Blue Square Bet South outfit following a Jon Garcia corner.
Jamie Lowry threatened for the Pilgrims with a long-range effort in the 40th minute but it was comfortably collected by Dorchester goalkeeper Jason Matthews.
The Magpies edged in front in the 49th minute when long-serving defender Mark Jermyn nodded a left-wing cross into the path of Gosling who fired home from close range.
This lifted their spirits and they enjoyed a dominant spell with Jamie Gleeson and Garcia both going close before holding on to reach the second round.