The U's twice hit back from behind through Tom Craddock and Harry Worley after Darryl Duffy opened the scoring for Cheltenham and Keith Lowe put the home side 2-1 up at the break.
James Constable's first goal since February then gave the visitors a second-half advantage and Peter Leven rounded off the scoring with a penalty 10 minutes from time.
"We were a bit suspect defensively regarding our organisation with the back four we had to chuck together," Wilder said. "But going forward, we caused them just as many problems as they did us in the first half, when it was attack, attack from both sides.
"In the second half we tightened up and it was great to see Andy Whing back from injury, which is a great boost.
"It was good that Peter Leven and Harry Worley have got through the game and James Constable has got a goal, so they are all big positives. Cheltenham are a decent side and it was a big examination, but it was a game we wanted to win and we've gone and done it."