Will Grigg's first goal since October proved decisive for the battling Saddlers, who had already ensured survival last week, and denied the Dons a fourth-placed finish in League One.
Gorman was given a guard of honour before the match following his decision to retire from football at the end of the season. But the 62-year-old was not happy after the game as the club's fans booed the team off and then left in their thousands before the team's annual lap of honour.
Gorman said: "It was emotional but I just wanted to get on with the game and it's turned out to be a very bad game for us. It's been our worst performance of the season, which is football - you've just got to accept it sometimes.
"I have to admit this because I'm leaving the club, I'm very disappointed in a lot of the fans leaving us at the end there when we needed them. When you play well it's easy getting behind us. We hold our hand up and say we didn't do well but we needed a lot of support and we didn't get it.
"The reason we went out there afterwards was for the chairman and all the young fans and for our manager because we've had such a good season. But we need support like the big clubs - like Huddersfield, Sheffield Wednesday and Sheffield United - that's what's probably got them over the line."
