Home captain Manuel Pascali was controversially sent-off after four minutes, for a two-footed challenge on Nigel Hasselbaink.
Murray Davidson slotted the away side ahead on the half-hour mark, however the hosts battled back and grabbed a deserved equaliser seven minutes from the end when Michael Nelson headed home from James Dayton's free-kick.
The game looked set for a stalemate but Vine fired home on 88 minutes to clinch all three points for Steve Lomas' side.
Referee Stevie O'Reilly infuriated the home support after dismissing Pascali for what he deemed to be a two-footed challenge on Hasselbaink after just four minutes.Killie boss Kenny Shiels was enraged at the decision and was sent to the stands after a heated debate with fourth official Andrew Dallas.
Hasselbaink was feeling the wrath of the home supporters every time he touched the ball, however the effects of Pascali's challenge became evident as he hobbled off on the half-hour mark to be replaced by Vine.
The visitors made the breakthrough two minutes later when Tade laid the ball to Davidson and the midfielder slotted low past Cammy Bell for his fifth goal of the season.
Alan Mannus was forced into his first save of the match eight minutes after the break when he parried away Paul Heffernan's powerful strike.
St Johnstone replied at the other end and Craig's dipping effort sailed just wide of Bell's right-hand post.
Saints continued to look the most likely to add to the scoring and Craig ended an excellent breakaway by flashing an effort just wide of the target after 58 minutes.
Dave MacKay was booked was a hauling back Harkins on 65 minutes and he was followed into the book by MacLean two minutes later.
Shiels introduced Danny Racchi and Dayton in place of Jeroen Tesselaar and Perez with 10 minutes remaining and the hosts reaped the rewards almost instantly.Dayton's deep free-kick found Nelson who angled a superb header beyond the despairing Mannus.
Racchi saw his effort deflected wide and Sissoko headed past the post as the hosts heaped on the the pressure in the closing stages.There was still time for another twist though and Vine charged through on goal before firing past Bell to ensure all three points for St Johnstone.