Portsmouth, in danger of dropping into the bottom tier of the Football League within five years of winning the FA Cup, had held out for an hour at the County Ground, but then Collins made his game-changing entrance.
The 22-year-old replaced Chris Martin and headed his first after just a minute on the pitch before making it 2-0 in the 66th minute.
Danny Hollands stretched the lead three minutes later, before former Shrewsbury frontman Collins scored again after 73 and 83 minutes.
The victory lifted Swindon to fifth place in the table, and Paolo Di Canio's side hold by a six-goal margin the best goal difference in the division.
They have eight points to make up on leaders Tranmere though, after the team from the Wirral won 1-0 at Bury thanks to Andy Robinson's left-footed strike in the 56th minute.
None of the other sides in the top six at the start of the day won, with third-placed Doncaster held 2-2 by second-placed Sheffield United at the Keepmoat Stadium despite going two goals in front.
The South Yorkshire derby looked to be going Doncaster's way when Billy Paynter converted an 18th-minute penalty and Dave Syers made it 2-0 after the break.
But United came back to make it 2-1 through Nick Blackman after 80 minutes.
And soon after Paynter was sent off, the 10 men conceded an equaliser in the 89th minute to experienced Blades striker Dave Kitson.
Brentford stayed in fourth after a goalless stalemate with Bournemouth, who fell two places to seventh, at Griffin Park.
MK Dons nudged up to sixth with a 2-1 win at Notts County, striking early through Darren Potter and Daniel Powell and then holding out after Jamal Campbell-Ryce's penalty narrowed the gap in the 56th minute.
Coventry suffered a 1-0 home defeat at the hands of Shrewsbury, for whom Marvin Morgan scored the only goal after 63 minutes.
Colchester's slide towards the bottom four continued with a 3-0 defeat at play-off hopefuls Crawley, whose goals came from Mark Connolly, Billy Clarke and Nicky Adams.
Crewe were 1-0 winners over mid-table rivals Carlisle, thanks to Lauri Dalla Valle's goal just before half-time.
And Preston won in similar fashion at Hartlepool, the only goal of their clash with the division's bottom side coming from John Mousinho in first-half stoppage time.
Second-bottom Scunthorpe led Oldham 2-0 with goals inside the opening 20 minutes from Niall Lanavan and Mark Duffy, but Cliff Byrne halved the advantage before half-time and Jose Baxter's penalty after the break earned the visiting Latics a 2-2 draw, despite James Wesolowski being sent off late on.
Jamie Paterson struck after 87 minutes to bring up a 1-0 home win for Walsall over Stevenage, and it was a day to forget for Leyton Orient manager Russell Slade whose current side lost 3-0 at the home of one of his former clubs, Yeovil.
Paddy Madden was twice on target for Yeovil, with James Hayter also scoring.
