After a goalless first half in front of an Easter Road full-house, David Smith had cancelled out James Collins’ opening goal and Hearts – battling for their Premiership lives – seemed on course for a point.
But McGhee inexplicably lunged in on Lewis Stevenson as he tried to shepherd the ball out for a goal-kick and Liam Craig slotted home the resulting penalty.
Callum Tapping had had the first sight of goal, forcing home keeper Ben Williams into an instinctive save from a close-range flick.
Craig’s lofted free-kick was headed against the bar by Jordon Forster as Hibs responded before Paul Cairney’s flick sprung Jason Cummings but he sclaffed his shot horribly across the face.
Stevenson screwed a shot over the top while Scott Robertson sent a diving header skidding wide of the target as Hibs began the second-half with purpose.
The pressure told when Collins (61) reacted quickest as Paul Hanlon’s knockdown fell kindly, lashing the ball into the roof of the net on the swivel.
Brad McKay had a glorious chance to level when he got first run at the near post to meet a corner, but his bullet header flew past the post.
Paul Heffernan was not far away with a flicked header but Hearts – under the cosh for so long – were soon level out of nothing, thanks to a dramatic intervention by Smith (72).
The Hearts substitute slalomed through the Hibs rearguard and although Williams parried his initial effort, Smith’s second attempt found the back of the net off Ryan McGivern.
But all battling Hearts’ good work was undone by 17-year-old McGhee’s rash intervention (83) and the home side held out with few alarms, Kevin McHattie’s stinging drive notwithstanding.