Two-goal Ashley Hemmings helped Walsall bring Swindon's four-match unbeaten run to an abrupt halt.
Sam Mantom was also on target as the Saddlers comfortably swept aside their play-off rivals.
"I thought the performance was lacklustre and I also think that we lost control, we didn't manage to gain a foothold in the game really," said Williams.
"A lot of credit to Walsall. They came here with a game plan and we weren't able to impose ourselves on the game like we should have been and ultimately they have come out on top.
"We sacrificed a defender for an attacking player to try to be more aggressive and win the game but that probably cost us with the goals.
"We have looked solid with three centre-backs, we wanted to change to be more aggressive but in hindsight maybe the change made it more difficult for us.
"We weren't at it, we looked lethargic and we didn't look like we were able to get a good spell of possession and keep the ball and start to impose ourselves on the other team.
"We looked like we were going to give the ball away cheaply so disappointing."