For 45 minutes the home side bossed the game and looked certain to add to the lead given to them by Ayegbeni Yakubu’s predatory finish.
But Keith Millen’s side were a different proposition after the break and drew level when Marvin Elliott bundled in from a corner.
In truth Sven-Goran Eriksson’s side were a shadow of their fluent selves after the break but stole a fifth straight win when Waghorn , on as a substitute, thrashed into the top corner.
There was an international feel to the line-ups with the two goalkeepers on display having been on opposite ends of a Euro 2004 penalty shoot-out, when Portugal’s Ricardo dumped Robins keeper David James and England out with the decisive kick.
James, 40, has had a disappointing season with Bristol City and should have been beaten inside 70 seconds as Leicester made a great start.
Richie Wellens played a quick free-kick out to the right where Kyle Naughton had found space.
His square ball was perfectly weighted for Yakubu but the Nigerian was wasteful as he fired wildly over the top from 10 yards out.
It was a let-off for the visitors and they struggled to get a grip on the opening quarter of an hour as the hosts popped the ball about confidently.
James made a fine save to his left to repel a fierce drive from Paul Gallagher as the Foxes turned up the pressure but the reprieve was only temporary as Yakubu gave the home side a deserved 21st-minute lead.
James made another smart stop to claw out a header from Sol Bamba but, when the ball was crossed back in, the former England keeper could only palm the ball out to Yakubu who made no mistake from six yards.
It was the Everton striker’s fourth goal in six games since starting his loan deal and surely the simplest.
Leicester looked threatening every time they attacked with Darius Vassell shooting wide and James saving bravely at the feet of Andy King after Yakubu had played a fine through-ball behind the defence.
The visitors had barely had a touch but started the second half brightly and Albert Adomah had already fallen over when in on goal before Elliott levelled in the 27th minute.
A Jamie McAllister corner was met by Elliott at the back post and, despite the efforts of Wellens on the line, the ball found its way in.
Leicester had been poor since the interval but snatched a win as the clock entered the five minutes of added time.
Steven Caulker could only hook his clearance into the back of King and substitute Waghorn was quick to latch onto the loose ball.
He let the ball bounce before driving an unstoppable shot which beat James at his near post and found the top corner.