An eventful match also saw home midfielder Johnny Ertl sent off in the second half while Johnny Mullins hit a post for the visitors.
Pompey, pre-season favourites to win the title, were roared on by the majority of the huge attendance and took the lead in the 25th minute when Patrick Agyemang headed in Andy Barcham's left-wing cross.
But Oxford were always dangerous on the counter attack and a mistake by Dan Butler let in Smalley for the equaliser after 35 minutes.
Portsmouth were still reeling four minutes later when Yassin Moutaouakil was caught in possession by Smalley who crossed for Potter to score in front of a record League Two crowd of 18,181..
Ertl was sent off for serious foul play just before the hour mark for a late tackle and things went from bad to worse for Guy Whittingham's team.
Potter got his second goal with a left-foot drive in the 64th minute and then set up Smalley for his second six minutes later.
Mullins almost got a fifth with a far-post header but his effort came back off a post.