Everton had led through Steven Naismith’s 57th-minute opener but the introduction of David Nugent and Jamie Vardy soon after sparked a turnaround as Vardy crossed for Nugent to score and then repeated the feat a few minutes later for Esteban Cambiasso to finish.
It looked like Nigel Pearson’s side were heading for victory until Romelu Lukaku’s header deflected off Upson and into the net with just two minutes left.
The result leaves the Foxes rooted to the foot of the Premier League table while Everton remain in 12th.
Despite the late comeback, there were still some boos at the final whistle from the home fans, who have not seen their side win at Goodison Park in the league for more than two months.
The visitors were largely the better side in a drab first half while Everton lacked any pace or purpose going forward.
They wasted the couple of chances they did get as Lukaku fired over the bar and Ross Barkley curled a free-kick over.
Andrej Kramaric came close at the other end when he curled a shot over while John Stones put in a last-ditch block to deny Schlupp in first-half injury-time.
Leicester should have taken the lead shortly after the break but Schlupp dragged his shot wide after Howard parried Matty James’ effort.
Instead it was Everton who went ahead after 57 minutes as Naismith cut inside from the left and finished at the near post.
Pearson replied by sending on Nugent and Vardy and the move soon paid dividends.
Vardy did well to get away down the right and when his cross was spilled by Howard, Nugent was on hand to finish from close range.
Lukaku missed a great opportunity to restore Everton’s lead minutes later as he failed to convert a one-on-one chance.
And the hosts paid the price as the Foxes soon made it 2-1. Again Howard was at fault as he failed to clear a cross and Vardy pulled the ball back for Cambiasso to finish.
The Everton fans groaned as Lukaku wasted yet another chance to score as the former Chelsea striker could only muster a weak shot after being played in by Naismith.
But the Belgian made a telling impact in the dying minutes when his diving header hit Upson and flew in to steal a fortuitous point for Martinez’s side.
Everton: Howard (4), Baines (5), Stones (6), Jagielka (5), Coleman (6), Besic (6), Barkley (5), McCarthy (5), Lennon (6), Naismith (7), Lukaku (5).
Subs: Atsu (6), Gibson (6), Mirallas (5).
Leicester: Schwarzer (5), Konchesky (5), Upson (5), Huth (6), Morgan (5), Simpson (6), Cambiasso (7), James (6), Mahrez (5), Schlupp (5), Kramaric (6).
Subs: King (5), Nugent (7), Vardy (7).
Man of the match: Jamie Vardy.
Sky Sports subscribers can sign in to watch extended highlights of Everton v Leicester from Goodison Park.