Deane Smalley netted a 76th-minute winner after Vale's Doug Loft cancelled out Alfie Potter's opener to make the final score 2-1 in Saturday's League Two clash at the Kassam Stadium.
"It's been a long time coming and the supporters deserve it," said Wilder, who also watched his side topple leaders Gillingham in midweek.
"We knew it would be a game that would open up, and that Port Vale would be on the front foot and coming at us and that suited us better.
"We defended very well when we had to and attacked well. We've now done well against the top two, and have to go again on Tuesday against Rotherham, another good team who are up near the top."
Wilder was delighted with matchwinner Smalley, who has just returned to the team following four months out with a groin injury.
"People can see what Deane's like when he's fit," he said. "I felt with his pace and with Alfie getting in behind, it was right to stick with that combination up front.
"Deane worked extremely hard out of possession and was good in possession. We've missed him for a lot of the season through injury and it's great to have him back."