After just 110 seconds the home side took the lead after Roland Lamah found the bottom corner from an acute angle to punish some slack West Brom defending.
The Swans had numerous chances to double their advantage, but were wasteful in front of goal, and were made to pay just after the interval.
After a frantic start to the second half, the Baggies grabbed a deserved equaliser thanks to an excellent Stephane Sessegnon strike from distance which put the visitors very much in the ascendancy.
And, after dominating for large spells of the half, the Baggies snatched a deserved winner, as Mulumbu bent the ball around a defender to beat a static Michel Vorm to seal an unlikely turnaround.
The victory moves the Baggies three points clear of the drop zone, but leaves the Swans looking nervously over their shoulders.
It was just the start that under-pressure boss Mel did not want as he went in search of his first win in charge, with the Baggies backline allowing Lamah in far too easily, giving him time to latch onto Wilfried Bony's pass, turn inside and find the bottom corner, without really being challenged.
The visitors' rearguard was in disarray early on, allowing Swansea several chances to increase their advantage.
Bony's close-range prod was well blocked by Ben Foster in the Baggies goal, before Ashley Williams wasted a great opportunity to score, heading into the ground and over unchallenged from eight yards out.
Victor Anichebe looked an isolated figure up front for Mel's men, with a solitary effort straight at Vorm the only real attempt at goal the Baggies had all half.
However, what Mel said at the break seemed to work, as West Brom came out of the dressing room with real purpose, and deservedly equalised just seven minutes into the half.
After good hold-up play from James Morrison, Sessegnon unleashed a ferocious effort from 20 yards which flew into the bottom corner to level proceedings.
It was all West Brom after that, with several last-ditch blocks from the beleaguered Swansea backline keeping the scores level.
Jonathan de Guzman did have a real chance to put the home side back in front, but his low strike was well saved by Foster with his legs.
However, the Baggies were not to be denied their first away league win in 11, after Mulumbu picked up the loose ball, carried it forward and curled a low, precise finish past a stranded Vorm to snatch all three points for the visitors.
The West Brom players' and staff reaction said it all at the end as they celebrated with their travelling support, with Mel looking especially relieved to have finally got that elusive first win under his belt.