The game's only goal came eight minutes after the break courtesy of James Collins, who weaved his way through the U's defence before slamming a low shot past Sam Walker.
But Dunne felt his charges deserved at least a point and said: "I don't agree that it was the right result.
"When they got the goal we had to go for it and the game became a lot more open.
"It was a scrappy game but I thought we were well in it and I thought we matched them. We had chances to score and we forced corners so I think overall we're disappointed we didn't come away from the game with at least a draw.
"Swindon's goal came against the run of play and we had a corner kick that got cleared and we switched off for a moment from a throw-in. That cost us because we need everybody concentrating and unfortunately at that moment we didn't and they scored.
"Once they went ahead they made it very difficult for us and we couldn't break them down.
"We threw a couple of forwards on and went for it but I thought we did well."
