After a slow start Aldershot settled into the game as the first half went on and Danny Rose headed just over the bar after captain Ben Herd whipped in a cross from the right in the 18th minute.
The hosts continued to push forward and in the 32nd minute Rose found Oliver Risser with a fine cross and his flicked header forced an excellent fingertip save from Michael Poke.
Torquay responded positively and two minutes later came close at the other end when Rene Howe beat Jamie Young but his downward header hit the post and was cleared.
Danny Hylton, on for Jeff Goulding at the break, almost made an impact eight minutes into the half when Rose's wide free-kick bounced in the six-yard box but the striker's scuffed shot was saved by Poke.
But after a fast-paced opening both teams failed to create any clear-cut chances until Risser smashed an effort against the crossbar from 30 yards with Poke well beaten.
The only goal of the game eventually came in the 83rd minute, courtesy of Shots defender Brown.
Substitute Peter Vincenti's header struck the bar and from the rebound Brown headed past Poke from close range to give the home side a crucial three points.
