The striker reacted quickest to a poor Jake Cole clearance to complete a five-minute turnaround for the visitors.
Plymouth's Paris-Cowan Hall opened the scoring just after half-time when he completed a superb downfield move to grab a 49th- minute goal.
Cowan-Hall - on as a first half replacement for Alex MacDonald - converted from 12 yards after being put in on goal by Warren Feeney's superb pass on the spin.
Albion turned the match around in five minutes as central defender Nathan Stanton headed home a 74th-minute leveller, with Plymouth failing to clear their lines from a corner. Keeper Cole flapped at visiting skipper Zander Diamond's header and Stanton swooped to restore parity.
Albion grabbed a winner against the run of play when Cole punched out Chris Palmer's cross and Weir headed home.