A fast start from both sides saw Ryan Stevenson and May have a brace of early chances for their respective sides.
The two sides continued to exchange blows but Steven Anderson's 21st minute challenge on Dale Carrick was deemed too fierce in referee Brian Colvin's opinion who sent the defender off.
It seemed only a matter of time before Hearts would take the lead, but against the run of play the home side won themselves a penalty when Nigel Hasselbaink was brought down in the area by Kevin McHattie.
May made no mistake from the spot and despite St Johnstone suffering further disruption with Murray Davidson leaving the field due to injury they took a 1-0 lead into half-time.
Hearts would have been optimistic of getting back into the game in the second half but less than four minutes after the restart they found themselves even further behind the 10-men.
Some neat play from Hasselbaink saw him play May in who made no mistake from close-range to notch his 18th goal of the season.
Hearts fought their way back into the contest when Carrick headed home a McHattie corner just after the hour mark.
But four minutes later May converted another penalty to claim his hat-trick following a McHattie handball to seemingly put St Johnstone on their way to their second consecutive victory.
In the 89th minute Hearts looked to have got a consolation when Sam Nicholson struck from close-range.
As Stevenson tried to retrieve the ball to restart the game quickly he became involved in altercation with St Johnstone goalkeeper Alan Mannus, which saw the pair sent off and St Johnstone reduced to nine men.
The home side had already made all three substitutions so left-back Tam Scobbie went in goal and within seconds he saw the ball rattle off his crossbar from a Callum Peterson header.
But there was still time for a late twist as Hearts captain Danny Wilson rose at an injury-time corner to head home and claim a point for his side who broke a three-game losing run.