It was the Stags' sixth straight League defeat and a 10th League game without a win, but a massive boost for an ailing Shrimps side who began the day with just one win in eight and were trailing to a Ross Dyer goal at the break.
There were few chances in the early stages, Ollie Palmer shot wide from 20 yard in the home side's only real threat.
But Dyer's goal provided the breakthrough for the home side in the 17th minute and it came from a superb curling pass from Chris Clements from the centre-circle, dropping perfectly into Dyer's path for the striker to beat Barry Roche with a low 15-yard finish.
Sam Clucas was too high from another splendid Clements pass and the visitors might have grabbed an unlikely equaliser right on the stroke of half-time as Stewart Drummond met a headed Martin Riley clearance with a 20-yard volley that zipped over the bar.
But Morecambe made a double change at the break and within six minutes one of the substitutes, Redshaw, had levelled it.
Lee Beevers tried to see the ball out for a goalkick but was caught out by the challenge of Stewart Drummond, who then drilled in a low cross that Redshaw turned home from close range.
Fellow sub Andrew Fleming screwed a good low chance wide three minutes later. Then Ellison dipped a volley over the home bar two minutes before Roche got down to save Palmer's low finish from Dyer's pass.
Roche was then the hero and the villain with 15 minutes remaining. His lunge brought down Junior Daniel in the box to earn a booking and concede a penalty.
But Roche guessed correctly to go full length to his left and block Sam Clucas' spot kick.
Instead it was the visitors who grabbed an 85th minute winner. Redmond was the architect with his run down the right and low cross that Ellison lifted over everyone under the right angle from 12 yards.
He was injured in the process and, having used their three subs already, they had to play out the game with 10 men and were grateful to Roche who denied Clucas with his leg with the final kick to preserve the win.