Swansea missed the chance to move clear of trouble at the wrong end of the table and could find themselves dragged right into a relegation battle after a fourth Premier League game without a win.
After taking an early lead, Swansea saw West Brom come from behind to earn their first win under Pepe Mel to boost their own survival hopes.
Vorm blamed his side's second-half performance for the defeat as they failed to build on their first-half lead.
"For me we can have no excuses," Vorm told the South Wales Evening Post.
"I was really looking forward to this game and after the first half I was thinking it was going to be three points and a big win for us.
"But if you don't give the same mentality in the second half and at least match what West Brom did then you will lose games.
"It does not matter who you play against you have to fight for the points.
"We may have been comfortable in the first half but you cannot afford to sit back. You saw what happened here.
"I hope we learn from it.
"There are still nine games to go but this is very disappointing."
Vorm believes the only silver lining from the game was the return of key man Michu after three months out following ankle surgery.
"That's the only positive from the game," added Vorm. "Let's hope he can get back to 100 per cent because we will definitely need him for the last nine games."