The Magpies were on course to leave with just their second point in six games when Alan Sheehan made it 2-2 just after the hour but Jennings lashed home the winner in the 84th minute.
Ronan Murray put County 1-0 ahead in the 14th minute before two quick goals from Ryan Lowe and Jason Koumas gave Rovers the lead and Sheehan missed from the spot - all before half time.
The visitors drew first blood when Tranmere failed to clear a Sheehan corner and Murray fired low past Owain fon Williams but Rovers were not behind for long.
Koumas fired home a great strike from 20 yards in the 25th minute before Lowe hit a Liam Ridehalgh cross into the roof of the net three minutes later.
Gary Liddle was body checked by Ash Taylor in the box moments later but Sheehan saw his spot-kick saved by the legs of fon Williams.
Sheehan made amends though, equalising with a superb 30-yard free-kick, but Jennings denied the Magpies a point with a fine strike from distance at the death.