Upson struck first with a perfect right-foot shot into the top corner 20 minutes from time after Tom Parkes had been penalised for a foul on Yeovil substitute Gozie Ugwu 25 yards out.
And the midfielder did it again in the closing minutes, catching Rovers goalkeeper Sam Walker by surprise with another superb strike from a wider angle. Sam Foley completed the scoring after racing clear in injury time.
Another good performance from Gary Johnson's League One high-flyers was marred when Jamie McAllister was shown a straight red card in the 73rd minute following an off-the-ball incident at a Rovers throw-in. Yeovil hit the post twice through shots from Luke Ayling in the first half and Reuben Reid in the second.
Rovers had the ball in the net twice but both efforts were disallowed and they have now failed to score in three home games this season.
David Clarkson almost pulled a goal back at the death, but his low shot was saved by Marek Stech.