Villa twice looked set for victory after a John Pantsil own goal and Kyle Walker's first ever league goal put them in the ascendancy.
But Andy Johnson and then Dempsey struck back for Mark Hughes's side.
Villa dominated the first half with winger Stewart Downing causing plenty of problems and linking up well with full-back Walker down the right flank.
Newcomer Jean Makoun also impressed in the middle of the park and Villa deservedly led through the unfortunate Pantsil.
He had also scored an own goal in the corresponding fixture last season.
But it was a different story in the second half as Villa lost momentum and the visitors grew in confidence with skipper David Murphy impressive in the middle of the park.
Villa were first to settle and Downing made a powerful run into the Fulham box and saw his low shot parried by Mark Schwarzer.
Record Villa signing Darren Bent was alert to tuck away the rebound but he had strayed marginally offside.
Downing was looking sharp and it was his quality which led to a Pantsil own goal giving Villa the lead after 13 minutes.
Makoun found the England winger on the right flank and he cut inside before delivering a teasing centre with the outside of his left foot.
Pantsil, under pressure from Gabriel Agbonlahor, could only head the ball past Schwarzer into the corner of the net.
Former Villa midfielder Steve Sidwell had Fulham's first shot on goal after 18 minutes but Brad Friedel was equal to his swerving drive from 20 yards.
Downing was causing constant problems down the right flank with his runs and testing crosses.
A mistake by Villa skipper Stiliyan Petrov allowed Mousa Dembele a free run at the penalty area.
But, after the ball broke to Duff, Villa left-back Ciaran Clark was alert to block his low shot.
Sidwell was only just wide of the target with a low shot which had Friedel scrambling across his goal.
Another error from Petrov played Makoun into trouble and Mousa Dembele tried to take advantage but James Collins made a typical block.
Sidwell brought danger to his side when he brought down Petrov 30 yards out in a central position.
Downing's shot was blocked but Makoun's follow-up effort was only just wide.
Villa began the second half on top and Schwarzer held on to a drive from Agbonlahor.
Bent then failed to make proper contact with a shot after making good ground.
Sidwell was providing the main threat for Fulham and Collins deflected his low attempt for a corner.
But after 52 minutes former Birmingham striker Andy Johnson brought Fulham level.
Friedel parried another shot from Sidwell - and Johnson was first to react to the rebound.
Clark could have restored Villa's lead immediately but failed to make clean contact with his header from Downing's centre.
Agbonlahor then teed up Young but he sliced his shot wide.
Young was yellow carded for a challenge on Sidwell after 62 minutes.
After 72 minutes a long-range strike from Walker restored Villa's lead.
Walker had time to line up a 30-yard shot which flew past Schwarzer into the corner of the net.
Petrov was booked for a foul on Sidwell with 13 minutes remaining and 60 seconds later Fulham levelled for the second time.
Friedel came out for but never reached Murphy's corner and Dempsey headed home.
Tempers became frayed and Makoun was yellow carded for a challenge on Dempsey and then Baird for chopping down Young.
Friedel made a fine save to block Johnson's powerful drive with a minute remaining.