Steven Naismith sparked an impressive comeback after Scotland had fallen behind to David Villa's penalty and strike from Andres Iniesta, before a Gerard Pique own goal set the home side on course for a precious point.
But glorious failures are Scotland's speciality and Fernando Llorente came off the bench to have the final say and deny Craig Levein's men a draw from a gutsy display - with Steven Whittaker sent off late on.
Scotland made three changes from the side which lost to Czech Republic on Friday night, with Phil Bardsley handed his international debut in place of the injured Alan Hutton after switching his allegiance to the Scots.
Kenny Miller - the subject of so much debate last week - was back in the starting line-up, along with Lee McCulloch, who missed the last game through injury and suspension. Gary Caldwell and Jamie Mackie were named on the bench.
Levein declared beforehand this Spain side could possibly be the best team Scotland have ever faced and one look at the team-sheet backed that theory, with nine of the players who started the World Cup final in tonight's line-up.
Xabi Alonso was back after recovering from flu, while Llorente's two goals against Lithuania at the weekend were not enough to prevent him being dropped to the bench.
It was not long before Spain were demonstrating their class, with a lovely long ball from Alonso collected by Villa - who lashed just over the crossbar. The Barcelona striker had another chance with a free header from David Silva's cross but Allan McGregor gathered.
Silva was then denied by the legs of McGregor as the home side soaked up the pressure from their visitors.
Scotland showed they were capable of posing a threat when Darren Fletcher raced down the left flank before cutting-back for James Morrison and he was only denied the opener by a last-gasp clearance from Joan Capdevila.
McGregor had earned plaudits for his performance in Prague and it looked like it could be another busy night for the Rangers number one, who did well to keep out Villa's strike from six yards.
Alonso then lobbed into the path of Silva but he failed to connect properly and miskicked from close-range, with the ball trundling wide.
Stephen McManus was the hero with a late winner here against Liechtenstein and he came to the rescue again when he hooked Iniesta's volley to safety on the six-yard line - only for Scotland to suffer a crushing blow a minute before the break.
Referee Massimo Busacca ruled that Whittaker handled a Sergio Ramos shot in the box and booked him for handball, despite the full-back claiming the ball had hit his shoulder. Villa converted from the penalty spot, with McGregor guessing right but failing to claw the shot past the post.
Hopes of a comeback in the second half suffered a setback when Spain increased their lead 10 minutes after the restart.
Santiago Cazorla rifled a shot off the legs of McManus and the rebound fell for Iniesta to despatch a low shot into the net from 15 yards.
Yet Scotland pulled a goal back just three minutes later when Naismith threw himself in front of Miller's
cross from the right to send a diving header past the grasp of goalkeeper Iker Casillas.
And Levein's men were back on level terms after 66 minutes, Pique prodding high into his own net as he tried to cut out a centre from Morrison.
Spain restored their lead with 11 minutes to go when substitute Llorente collected a long cross from Capdevila in plenty of space and was allowed to stab home from close range to maintain the world champions' 100% record in Group I.
Scotland - and Whittaker's - misery was compounded when he picked up a second yellow for a challenge on Ramos with a minute to go.