Peter MacDonald broke the deadlock for St Johnstone after seven minutes before Liam Craig doubled the advantage with a cool finish after 21 minutes.
Jon Daly pulled a goal back for United seven minutes after the restart before Danny Swanson restored parity with a sublime volley from 25 yards.
United substitute Jennison Myrie-Williams sent the home crowd wild when his deflected shot found the back of the net after 72 minutes.
But Kevin Moon completed the scoring a minute later to give Saints a share of the spoils.
Darren Dods stepped in for the injured Andy Webster in United's only change from the side that thrashed Falkirk 4-1 on Sunday.
On-loan striker Cillian Sheridan was handed his St Johnstone debut after being ineligible to play against his parent club Celtic on Sunday.
The visitors made four changes in total with Moon, Steven Anderson and MacDonald also starting. Graham Gartland missed out following his red card in the 4-1 defeat to the Hoops, strikers Steven Milne and Kenny Deuchar both dropped to the bench and Murray Davidson missed out because of injury.
The visitors opened the scoring after only seven minutes when Mihael Kovacevic failed to deal with a Jody Morris cross and Moon headed the loose ball into the path of MacDonald who volleyed the ball past Dusan Pernis.
MacDonald could have grabbed a second three minutes later when Sheridan laid the ball neatly into his path but the striker's shot on the turn fizzed wide.
St Johnstone goalkeeper Graeme Smith was called into action for the first time after 15 minutes when he comfortably saved a dipping Craig Conway free kick from 25 yards.
Daly then tested the Saints keeper with a low drive from the edge of the area after 20 minutes which Smith gathered at the second attempt.
The visitors doubled their advantage after 21 minutes when a Danny Grainger throw-in caused panic in the United defence with MacDonald's flick falling kindly for Craig who slotted the ball home from an acute angle with a tidy finish.
The home support were screaming for a penalty in the 26th minute when Swanson fell in the box under pressure from Grainger but referee Stevie O'Reilly waved play on.
With United's defence again in disarray, Moon should have scored a third but the midfielder dragged his shot wide of the target from 12 yards with 29 minutes played.
The home side pressed for a route back into the game with Smith having to look lively to fist away a David Goodwillie snap-shot before the break.
A quick throw from Paul Dixon gave Daly an early opportunity to have a shot at goal but again Smith comfortably gathered the Irishman's drive.
But Daly did pull a goal back for United after 52 minutes when he headed home Conway's pinpoint free kick.
And the striker thought he had equalised a minute later when he netted from the edge of the area only for the effort to be ruled out for offside.
Daly then supplied the ammunition for Prince Bauben but the Ghanaian lashed the ball into the side-netting from a tight angle.
United introduced Myrie-Williams and top scorer Damian Casalinuovo for Conway and Goodwillie as they looked to restore parity.
Swanson, who had been thriving in the centre of midfield, conjured up an equaliser with an exquisite volley from 25 yards that gave Smith no chance to set up a rousing finish.
And the comeback looked to be complete after 72 minutes when substitute Myrie-Williams cut inside from the right wing and his shot took a cruel deflection off Grainger leaving Smith helpless once again.
However, there was life left in St Johnstone yet as they got back on level terms a minute later when substitute Deuchar nodded the ball into the path of Moon who swivelled before passing the ball into the bottom right corner.
Bauben could have wrapped up the points for United but Smith pulled off an excellent save to prevent the hosts from their fifth consecutive victory.
Tempers boiled over deep in injury time following a late Deuchar challenge on Pernis which saw Morris and Dixon cautioned.