Paul Hartley scored his fifth penalty of the season to give the hosts the lead but Jamie Mole equalised after 24 minutes before Gregory Tade added a second seven minutes before the break.
However, a second-half double from in-form striker Scott Vernon gave the Dons a hard-fought win to avenge their humiliating fifth round replay defeat to the Irn Bru First Division side in the Active Nations Cup in February.
Mark McGhee made one change from the side which drew with Motherwell with left winger Mitchel Megginson replacing defender Nikola Vujadinovic.
Rovers boss John McGlynn also made one change from the side which drew 0-0 with Dundee, with Mole preferred to John Baird.
Raith travelled to Pittodrie on the back of an impressive eight-game unbeaten run and had not conceded in their previous five matches, but could have gone behind after only four minutes had Andrew Considine managed to direct his header on target from a tantalising Paul Hartley free-kick.
However, the hosts did break the deadlock after 10 minutes from the penalty spot.
Dons skipper Hartley coolly converted from 12 yards after Willie Dyer had crudely ended Rory McArdle's surging run into the box.
Raith were gifted an equaliser after 24 minutes when Jamie Langfield, who had endured a nervy start to the match, made a mess of collecting a Stephen Simmons free-kick and the grateful Mole calmly slotted the loose ball past the stranded goalkeeper.
Derek Young fired wildly over the bar before Mark Ferry was guilty of spurning an even better opening at the other end when he failed to apply the finishing touch to Tade's enticing centre.
But it was Tade, who scored the winner when the sides met back in February, who gave his side the lead when he tapped home after Mole capitalised on a Jerel Ifil error.
Raith should have extended their lead four minutes after the restart but the unmarked Laurie Ellis could only head Allan Walker's driven corner over the crossbar.
Rovers were made to rue that miss when the Dons drew level after 56 minutes when Vernon headed home a Hartley free-kick.
The home side looked the more likely to add a third with Chris Maguire flashing a shot wide of the target and substitute Vujadinovic, who had replaced the injured Considine, heading over.
The hosts suffered a blow with 15 minutes remaining when skipper Hartley limped off with a hamstring injury and was replaced by Ryan Jack.
Despite the loss, Aberdeen regained the lead after 77 minutes when Young exploded down the wing and played the perfect ball across goal to give Vernon the easiest of finishes to the delight of the home support.
Darren Mackie, who had replaced the ineffectual Megginson at the break, should have made certain of the result in the dying stages but failed to connect properly with only the goalkeeper to beat.