Kris Commons drove the champions into the lead in the third minute but Saints striker Gregory Tade levelled in the 18th minute before the visitors were denied what looked a clear penalty when Steven Anderson challenged Commons inside the box.
But Celtic were well off the pace all afternoon and in the 80th minute they paid dearly when former QPR striker Vine curled the ball in from the edge of the penalty area to hand the Perth club their first win of the league campaign.
It was not the result or performance Neil Lennon was looking for ahead of their Champions League opener against Benfica at Parkhead on Wednesday.
However, the result will delight St Johnstone manager Steve Lomas who had seven players down with a virus on Thursday, which led him to cancel training.
Despite the sickness, he was able to field the same side which drew 0-0 with Dundee United in their last outing, with Vine supporting Tade in attack.
Celtic's new Venezuelan signing Miku was handed his debut while Lennon's other new outfield recruit, Lassad, started on the bench.
Skipper Scott Brown shrugged off an ongoing hip problem to return to the midfield, with Charlie Mulgrew back in defence after recovering from an Achilles injury.
It looked like it was going to be a stroll in the sunshine when Gary Hooper set up Commons 20 yards from goal and watched as the Scotland midfielder drilled a shot low past Alan Mannus and into the far corner of the net.
In the seventh minute, with Celtic in command, Commons had another effort from a similar distance but this time Mannus clutched the ball comfortably.
The Parkhead side then relaxed their grip, allowing Saints back into the game.
After Mulgrew was penalised 25 yards from goal for going through the back of Tade, Celtic goalkeeper Fraser Forster had to dive high to his right to prevent Dave Mackay's free-kick dipping in under the bar.
Hooper and Victor Wanyama had shots which flew wide of the target, but in the 15th minute Celtic defender Kelvin Wilson had to throw himself to block a goalbound shot from Peter Pawlett. The Saints wideman had cleverly worked a one-two with Murray Davidson before driving into the visitors' box.
Three minutes later Wilson was less vigilant, as was Mulgrew, when Vine played the ball into Tade, who turned and drove low past Forster from the edge of the box.
There was controversy in the 26th minute when Anderson, chasing his own header to Mannus which was short, tackled Commons inside the box without getting the ball but referee Euan Norris turned a deaf ear to a vociferous and valid penalty claim.
Just after the half-hour mark Miku drove high over the bar from the edge of the box when he had the chance of debut glory.
However, a minute before the break Pawlett missed a much better opportunity when he put the ball over the bar from 10 yards after being set up by Tade.
St Johnstone midfielder Liam Craig had a decent chance five minutes into the second half when he was played in by Mackay but his drive from 16 yards sailed over the bar.
Lennon refreshed his side in the 58th minute by bringing on teenage striker Tony Watt for Miku but seconds later it was Commons who went close to restoring Celtic's lead with a powerful 30-yard drive which just cleared the bar.
Defender Thomas Rogne then replaced Emilio Izaguirre, booked moments earlier for a foul on Pawlett, before there was a penalty claim for the home side when Mulgrew, who had moved to left-back, knocked Pawlett off the ball in the box.
Celtic began to enjoy more possession and in the 65th minute Mulgrew, with a delightful pass, set up Commons who squared the ball to Hooper but his shot was blocked by Mannus who eventually smothered.
The game stretched further with Tade's power and strength proving a handful for the Hoops defence while Celtic's passing game was too erratic for the liking of Lennon.
Nigel Hasselbaink replaced Pawlett in the 77th minute before Lassad came on to make his debut at the expense of Brown.
It was the home side's substitution which paid off when Hasselbaink played the ball to Vine who neatly stepped away from Rogne before curling a wonderful shot past Forster and into the far corner.
Celtic's response was predictable but Mannus made a point-blank save from Watt in injury-time to ensure an unlikely but welcome win for the home side.