Steve Jennings' fine strike in the 84th minute handed Rovers all three points after the Magpies had first broken the deadlock and then battled back to make it 2-2 just after the hour.
Ronan Murray fired Notts County in front in the 14th minute but two goals in the space of three minutes from Jason Koumas and Ryan Lowe put paid to their fine start.
Alan Sheehan then missed a penalty on the half hour - although he made amends after the break with a fine free-kick to make it 2-2 for League One's basement boys.
Jennings had other ideas though and a fine late strike from 30 yards settled it with McMahon insisting Tranmere never stopped believing they could claim all three points.
"We started well and I kept saying to the boys you need to capitalise and score when you're having good spells like we had and keep it tight when they had good spells," said McMahon.
"I felt we needed to score in the first 15 minutes but they did - although I do think we were unfortunate to go behind like we did.
"But we showed a good mental attitude, good strength and good character to bounce back from that because we were on top and going behind can have an adverse affect.
"I didn't think it did this time - we kept believing in what we were doing out there and obviously we got our rewards from that and got the result.
"I thought the fans were fantastic - they stuck with the players, they got behind them and I think everyone got the rewards for the good performance all round and now we move on."