It was a big financial blow for the Midlands club and also weakened caretaker boss Kevin MacDonald's chances of being given the job permanently.
Gabriel Agbonlahor and Emile Heskey netted for Villa but Rapid refused to buckle and goals from Atdhe Nuhiu, Mario Sonnleitner and Rene Gartler stunned the near 30,000 crowd.
The hundreds of Rapid fans held up a 'your nightmare returns' banner before kick-off - a reminder to Villa of their victory at the same ground 12 months ago - and it rang true as Stiliyan Petrov also missed a second-half penalty.
Villa made a promising start in the driving rain and swirling wind and Agbonlahor came close to breaking the deadlock after 60 seconds.
Ashley Young threaded a pass through to Agbonlahor, whose left-footed shot was well struck. But Rapid keeper Raimund Hedl was equal to the task and held onto the low drive at full stretch.
The Rapid fans were making their presence felt with a constant barrage of noise but James Collins almost silenced them after 17 minutes.
Young's perfectly flighted corner picked out the Welsh international but his header flew just wide.
Hedl then parried away a dipping Young free-kick and Collins had a close range effort blocked.
After 22 minutes Villa made their supremacy count with Agbonlahor's first goal of the season.
The impressive Young was the provider with a teasing low ball across the box and Agbonlahor made no mistake from close range.
It needed a fine tackle from Markus Heikkinen to deny Stephen Ireland as he prepared to shoot.
Agbonlahor tried to continue after suffering a knock to his leg and he was replaced after 40 minutes by Marc Albrighton.
Albrighton quickly made his presence felt with a powerful run which took him past three challenges before he drilled his eventual shot just wide.
But after 52 minutes Villa were stunned when Nuhiu brought Rapid back level on aggregate at 2-2.
Beye lost possession on the left flank and was then turned past by Veli Kavlak.
His eventual cross was headed home at the near post in powerful fashion by Nuhiu who had also netted in the first leg a week ago.
Villa came close to regaining the lead after 61 minutes.
Heskey's cross was diverted into the path of Young and the ball rebounded off him and inches wide of the far post.
Collins was just wide with a header from Albrighton's centre but the contest was now more evenly matched.
Villa had strong claims for a penalty rejected when Young appeared to be brought down by Markus Katzer.
Referee Eric Braamhaar pointed to the spot when Heskey was fouled by Sonnleitner after 72 minutes.
But skipper Petrov's spot-kick was saved to his right by Hedl before Heskey wasted the rebound.
Villa regained the lead after 76 minutes when Heskey chested home a volley from Habib Beye.
But the home side were stunned as Rapid struck twice in the space of three minutes.
First Guzan could only parry a header from Sonnleitner which still contained enough power to cross the line.
Then in the 81st minute substitute Gartler converted a low centre from Christopher Trimmel.