With the game at the Hawthorns heading for a goalless draw, Southampton were awarded a last-minute spot-kick after Youssouf Mulumbu was adjudged to have felled Luke Shaw. Rickie Lambert converted it and left the Baggies ruing referee Kevin Friend's decision.
After watching replays of the incident after the match, Clarke remained adamant that his side had been harshly treated.
"I've seen it; it was very, very soft," he said. "There's no doubt in my mind that it's a very, very soft decision."
The former Chelsea and Liverpool assistant manager also felt that his side had done enough to earn at least a point from the game.
"Yeah, I think it was (a harsh result)," Clarke said.
"If you took the first half in isolation Southampton were better than us, if you look at the second half I think we had better control of the game, we looked the team more likely to nick a result although I thought it was going to peter out to 0-0.
"To lose so late in the game, to such a soft decision, is really disappointing."
The Albion boss chose to replace forwards Shane Long and Nicolas Anelka late on with the score poised at 0-0.
"We tried to freshen it up," explained Clarke. "Shane and Nico had toiled away all afternoon trying to get the break for us, we'd created some chances in the second half - certainly more in the second half than we did in the first half. We were in the game.
"First half I thought we struggled to get a foothold in the game, second half sure we were in the game. Like I said, we tried to freshen it up, we brought on a couple of fresh forwards to try and give that impetus and try and get the three points."
Clarke argued that the frustration of losing had only been added to by the nature of the defeat but insisted that he was still happy with his squad.
"It's disappointing to lose but to lose to a soft decision makes it extra disappointing. The fans will go away and they'll be disappointed. They'll be disappointed with me, they'll be disappointed with the team, that's the nature of football," he added.
"They know that they've got a quality player in Nicolas as part of the squad but we've got a good squad of players, we've got other players who can come in and make a difference. It was nice to see Matej Vydra come in and make his debut and he looked really sharp."