Argyle defender Onismor Bhasera fouled Shots right winger Adam Mekki and Brown met Anthony Tonkin's pacy dead-ball delivery with a textbook downward far-post header to beat home keeper Jake Cole.
Dean Holdsworth's side forged further ahead when Josh Payne sent Cole the wrong way from a 53rd minute penalty following a foul on Shots striker Michael Rankine.
Plymouth midfielder Conor Hourihane came close to reducing the deficit two minutes later when he scooped a 20 yard shot just over the bar.
Hourihane also had a 49th minute effort blocked on the goal-line by Aldershot's Sonny Bradley.
Carl Fletcher's side did their utmost to restore parity in the first half. Argyle striker Nick Chadwick sent a 13th minute strike just wide from 25 yards and visiting keeper Jamie Young spilled a 23rd minute speculative effort from Andres Gurrieri.
