Late goals from debutant Jabo Ibehre and substitute Freddie Sears gave the U's their first win since November and helped them climb out of the relegation zone.
The Saddlers were bidding for a fifth straight win and had won six of their previous seven in League One.
Smith said: "I don't think (goalkeeper) Ian (McLoughlin) has made a save all game and a mistake leads to the first goal.
"It could have gone anywhere, but it falls to Jabo (Ibehre) and that opened the game up.
"We had to get something out of it, but they've ended up scoring a disputed second goal where we felt it was offside. All runs come to an end.
"We're really proud with the run we've been on and we've got to go again now.
"We had a reverse at Leyton Orient last month and we've got to bounce back now.
"We spoke about it at length before the game, about the first goal being important, but I can't really remember Ian making a save.
"I don't need to rant and rave because it was an even game. I don't think we deserved to win it and I don't think they deserved to either."
