After a goalless first half, St Johnstone came out after the break fired up and in the 54th minute, Graham notched his third goal in four games since arriving on loan from Dundee United.
Killie’s mercurial midfielder Alexei Eremenko was sent off in the 71st minute after picking up a second yellow card, making a hard task even more difficult for the Ayrshire men.
The inertia of the early stages ended in the eighth minute when a 25-yard drive by Eremenko was saved by Perth keeper Alan Mannus before the talented Russian-born Finn set up Killie striker Josh Magennis who drove wide from just inside the box.
Eremenko was behind almost all of Killie’s goal threats, his 19th-minute corner met by the head of defender Mark Connolly, whose close-range header drew a fine save from Mannus.
Saints were working hard enough and enjoying their fair share of possession but struggled to fashion anything resembling a clear-cut chance.
The high ball to Graham was their favoured option with one effort from the forward easily saved by Killie keeper Craig Samson.
Either side of that, Eremenko, booked by referee Steven McLean in the 29th minute for diving outside the St Johnstone box after going past Davidson, had further efforts.
He side-footed a cross from Tope Obadeyi just past the post from eight yards before rifling a powerful drive from 30 yards, again, just wide of the target.
But whatever Saints boss Tommy Wright said to his players at the interval seemed to work.
Samson made saves from Graham and skipper Dave Mackay before the former Raith Rovers striker put the visitors ahead.
As St Johnstone swarmed around the Killie box, Michael O’Halloran’s initial shot was blocked but when the ball eventually fell to the unmarked Graham he smartly hooked it high past Samson and into the net from 10 yards out.
When Eremenko was sent packing for pulling back O’Halloran after losing possession, it looked like a long way back for the home side.
Mannus made a fine save from Connolly’s header as the Ayrshire side rallied and Magennis managed to get the ball in the net from a Jamie Hamill corner but referee McLean immediately ruled it out for an infringement.
As the game stretched in the final minutes, Saints substitute Liam Caddis, on for Lee Croft, came close with a long-distance drive.
Although Kilmarnock tried their hardest to get the goal to take the game into extra-time, they ultimately came up short and will probably wonder what would have happened if Eremenko had stayed for the full 90 minutes.