Socceroos midfielder Marco Bresciano opened the scoring in the 17th minute with a thunderous volley from distance, before Jordan Rhodes scored his first Scotland goal in his first start in the 28th minute with a diving header from a Danny Fox cross.
Australia substitute Jason Davidson put through his own goal with a header from a Fox cross in the 63rd minute, five minutes after replacing David Carney to make his debut, before Scotland substitute Ross McCormack, who had replaced Rhodes, stretched the home side's lead in the 76th minute with a drive from the edge of the box.
It was an encouraging night for Craig Levein whose team will be buoyed going the qualifying double-header against Serbia and Macedonia at Hampden in September, and the Scotland boss might just have found a new hero in Rhodes.
The 22-year-old, who scored 40 times for Huddersfield last season, was thrown in after QPR forward Jamie Mackie pulled out through injury, and in the absence of Craig Mackail-Smith and Kenny Miller.
The striker almost got off to a dream start in the fourth minute when James Morrison got raced on to a Robert Snodgrass pass before swinging a cross into the six-yard box.
However, Rhodes failed to connect properly with an attempted back-heel and the chance was gone.
Three minutes later, in what was a fine start by Levein's men, the Huddersfield player headed an Alan Hutton cross from the right a yard past Mark Schwarzer's left-hand post.
It was encouraging viewing for the Tartan Army foot soldiers who had braved a rain-soaked Edinburgh night and in the 12th minute Steven Naismith tried his luck with a long-range strike only to see Schwarzer save down to his right.
A goal mouth melee in the 16th minute then left everyone confused.
With the Scotland defence sleeping the Aussies took a quick throw-in on the left with Robbie Kruse racing into the box.
His cross was blocked by retreating Scotland captain Gary Caldwell and the ball went over the line for what should have been a corner.
However, Norwegian referee Tom Hagen played on and the official surprisingly played on again when Brett Holman's shot from Kruse's second pass appeared to cross the line before being stopped by the arm of Fox, who then cleared for a corner.
The Socceroos complained long and loud to the referee but they became more excited but for a different reason when, from Kruse's corner which was cleared by Scotland defender Christophe Berra, Bresciano fired an unstoppable dipping volley from 25 yards past Allan McGregor.
Moments later, the former Rangers keeper hobbled off to be replaced by Blackpool's Matt Gilks, who was making his debut, before Morrison was replaced by Shaun Maloney.
After a good start it now looked rather bleak for the Scots but Rhodes brought the crowd back to life with a fine diving header from Fox's cross which flew past Schwarzer and into the far corner.
Scotland finished the first-half in control and looked in the mood at the start of the second period with some crisp and incisive build-up play.
However, in the 56th minute, after some slack defending by Hutton, it took a Gilks save to prevent Luke Wilkshire's shot from 14 yards going in under the bar, with the home side defending the corner.
Fox, impressing on his return to international football after three years in exile, was involved in Scotland's second goal when his cross to the back post from the left was inadvertently headed into his own net by Davidson, who held his head in his hands as the realisation set in.
A series of Scotland substitutions followed and it was Leeds' McCormack who made the biggest impact when he fired in from the edge of the box on a Scotland break to clinch the win.
In the 87th minute the introduction of former Hearts midfielder Ian Black to make his debut for Caldwell, was met by a loud chorus of boos from the home fans.
Levein had drafted the Rangers man into his squad after saying it would be difficult to pick players from the Ibrox club due to them playing in the Irn-Bru Third Division, and it provided a rather curious end to what was a good night's work for the national side.