Murray nodded in a free-kick from Darren Ambrose, the scorer of the opening goal, as Palace overturned the odds against a much-changed United team at Old Trafford.
Ambrose had ended shot-shy Palace's long goal drought with a stunning goal after 65 minutes but United forced extra time with a Federico Macheda penalty.
Murray may have been close to straying offside as he got forward for the winner but Palace deserved their success for their endeavour throughout a tight contest.
United may have made 10 changes but were hardly lacking in senior experience with the likes of Dimitar Berbatov, Park Ji-sung, Jonny Evans and Antonio Valencia starting.
Full-back Fabio da Silva was the only survivor from the weekend draw against Newcastle but Chris Smalling returned for his first start since October's infamous 6-1 thrashing by Manchester City after overcoming a foot injury.
Palace, backed by a sizeable following in a below-capacity 52,624 crowd, went into the game with confidence despite failing to score in their previous five games.
Manager Dougie Freedman made five changes himself but his team looked the brighter at the onset.
Wilfried Zaha was a recurrent danger and after blasting over he appealed for a penalty when he ran at Fabio and went down on the edge of the area - but referee Chris Foy saw nothing untoward.
Darron Gibson had a shot deflected behind for United but the lively Zaha went close again when he fired over from seven yards.
United began to settle and established a measure of control but Palace had another chance when Sean Scannell snatched the ball from Valencia and Kagisho Dikgacoi dragged a shot wide.
United were forced to make a change when Fabio limped off after 37 minutes and 19-year-old Ezekiel Fryers replaced him at left-back.
The hosts had their best chance of the first half when Valencia crossed from the right and Mame Biram Diouf volleyed over with an overhead kick.
Palace also had injury problems as Scannell was forced off for Jonathan Parr and Dean Moxey was receiving treatment when the first half ended.
Moxey was unable to continue and was replaced by Ambrose after the break while United boss Sir Alex Ferguson gave Ravel Morrison a chance in place of Berbatov.
United almost took the lead soon after the restart as Valencia robbed Anthony Gardner and hit a low shot past Lewis Price but it rolled narrowly wide. Valencia claimed the goalkeeper had got a fingertip to the ball but a goal-kick was given.
Morrison then tested Price with a long-range shot as United upped the tempo but, despite failing to grasp the ball first time, the keeper remained unbeaten.
Price saved from Morrison - but just as it seemed United were taking charge Palace took the lead with a stunning goal against the run of play.
There seemed little danger as Ambrose received the ball from Zaha but he launched a vicious shot from 35 yards which curled into the top corner.
It was a brilliant way to end a goal famine which had lasted 550 minutes but the London side's lead was wiped out just four minutes later - Macheda netting from the spot after his shirt was grabbed by Patrick McCarthy.
Palace refused to let their heads drop and had the better chances as normal time ran out.
Murray hit the side-netting in the last minute and Zaha shot straight at Ben Amos.
Gibson went close early in extra time when a shot from the right flew agonisingly wide after taking a slight deflection off Gardner.
Yet Palace proved again that they were made of stern stuff as they reclaimed the lead in the 98th minute.
Ambrose clipped in a free-kick from the left and Murray shook off Evans to head home from close range.
Gibson led the United response with another long-range effort which narrowly missed and Macheda poked an effort wide.
Ambrose almost hit a second screamer when he drilled a 40-yard free-kick at goal but Amos made a fine save.
The second period of extra time was a much cagier affair with Palace defending deeply but they survived a couple of heart-stopping moments to prevail.