McSheffrey, who had already converted a 21st-minute penalty, slammed home his second spot kick in the 92nd minute after Aidan White's foul on Alex Nimely to inflict a second straight defeat on his former club.
Leeds, for whom caretaker-manager Neil Redfearn was taking charge for the third time since the sacking of Simon Grayson, had earlier equalised before half-time through Ross McCormack, but after a third defeat in four games their play-off chances are fading fast.
Coventry created enough chances to have sealed the points inside 90 minutes, but were let down by wayward finishing and remain six points from safety.
Injury-hit Coventry recalled midfielder Carl Baker and defender Jordan Clarke in place of latest casualties Sammy Clingan and Richard Wood, while Leeds were unchanged from the side which started in the weekend defeat to Brighton.
Neither side had created a chance of note before referee Pat Miller deemed visiting midfielder Pugh's challenge on Coventry striker Nimely in the penalty area to be a foul.
McSheffrey, on loan at Leeds for the second half of the 2009/10 season, stepped up to place the subsequent spot-kick into visiting goalkeeper Andy Lonergan's bottom right-hand corner in the 21st minute.
Leeds forward McCormack had pulled a chance for the visitors wide from 20 yards in only the third minute, but he was on target for his 16th goal of the season 11 minutes later to haul his side level with a smart finish from inside the area.
Coventry then enjoyed their best spell.
Nimely dragged his effort wide from the edge of the area, Clive Platt's goalbound header was cleared by Luciano Becchio and McSheffrey's 20-yard effort was well saved by Lonergan.
McSheffrey was then inches wide with a blistering volley just before half-time.
Leeds enjoyed more possession after the break. Midfielder Adam Clayton's rising shot cleared the crossbar and after Baker fluffed a gilt-edged chance from six yards soon after, Leeds defender Aidy White forced Coventry goalkeeper Joe Murphy into a fine save with a shot from 12 yards.
The visitors were dealt a blow just after the hour-mark when midfielder Fabian Delph, on loan at his former club from Aston Villa, was withdrawn through injury and replaced by Michael Brown.
Chances dried up midway through the second period, but Coventry still looked the more likely side to break the deadlock.
Midfielder Gary Deegan lashed a 25-yard shot over the crossbar, Nimely's effort was blocked by Pugh and Oliver Norwood failed to get enough curl on his 30-yard free-kick.
Leeds sent on Ramon Nunez for McCormack in the 82nd minute before defender Darren O'Dea threatened to snatch the points for the visitors from close range.
But in the second minute of time added on White brought down Nimely in the area and McSheffrey took no chances with his power penalty as Leeds went down to a stoppage-time goal for the second time in four days.