The Moroccan's deflected strike from the edge of the box was his first for the Teesside club and ensured Boro claimed all three points.
Jamie Ward had put the hosts in front from the spot in the first half after he had been felled by Nicky Bailey in the area.
However, Seb Hines grabbed an equaliser for the hosts before Zemmama struck to help in Boro's fight against the drop.
In truth it was no less than they deserved as they completely dominated the tempo of the second half following a dour first period.
Although Boro remain just two places above the danger places, they are now six points clear of 22nd-placed Scunthorpe.
With Jason Steele and Danny Coyne both injured, Tony Mowbray gave a debut to Paul Smith, having signed the goalkeeper on-loan from Nottingham Forest earlier in the day.
The former Baggies boss was desperate to tighten up at the back having seen his side concede 15 times in their last five league matches - which included a 5-2 hammering at Reading over the weekend.
However, Boro's troubles have not been in front of goal as they have also found the back of the net on nine occasions in that time.
And it was Rams shotstopper Frankie Fielding who was forced into action early on as he did well to grab the ball at the feet of Scott McDonald who had burst through on goal.
That was about as entertaining as it got in the opening half hour but Boro were undone with Derby's first meaningful attack on goal in the 32nd minute.
Theo Robinson's dink over the top put strike partner Ward clean through on goal but, before he could strike, he was recklessly bundled over by Bailey, who rather fortuitously received only a yellow card.
Ward himself stepped up and powered his kick straight down the middle to leave Smith with no chance.
After being booed off at half-time, Boro came out all guns blazing after the interval, with Bailey firing over and Leroy Lita unfortunate not to be able to get a touch on Barry Robson's teasing delivery.
However, only an offside prevented them falling two behind.
Stephen Pearson burst forward on the left-flank and his cross left Conor Doyle with the simplest of finishes, but after heading home, the winger found he had strayed beyond the last defender.
Lita, who had notched four goals in his last five appearances, McDonald and Marvin Emnes all forced smart saves from Fielding, while Rhys Williams' header was cleared off the line.
Their pressure eventually told and Hines gave Boro a lifeline in the 74th minute.
McMahon's free-kick on the left channel was headed goalwards by Hines at the back post and although Derby's John Brayford got the final touch, it was awarded to Hines.
Backed by the boisterous crowd, the hosts sensed victory and only a magnificent save from Fielding from Robson's header kept the scores level.
However, it was all in vain as substitute Zemmama snatched a winner at the death.
After McMahon's ball into the box had been cleared to Zemmama on the edge of the area, the defender's deflected strike found the back of the net to the delight of the Riverside faithful.