The former Tottenham boss was the leading candidate to succeed Tony Pulis at Selhurst Park after Palace's initial top target Malky Mackay was removed from their shortlist.
But Sky Sports News HQ presenter Jim White has revealed he has spoken personally to Sherwood, who confirmed the Palace job was "not the right fit" for him currently.
White reported: "I've spoken to Tim Sherwood, who is in Spain at the moment, and he has told me that, after some careful consideration, he no longer wants to be considered as a candidate for the vacant manager's job at Crystal Palace.
"His representative has contacted Palace co-chairman Steve Parish to tell him just that, so Parish now knows that Sherwood has dropped out of the race.
"Sherwood told me: 'I wish them well with the hunt for a new manager, but it is not the right fit for me at this time. Crystal Palace is a great club with a great tradition, but they’re in a difficult situation which was unforeseen.'
"Sherwood thanked Parish for granting him an interview last weekend, and he felt the interview went very well, but he no longer wishes to be considered for the job."
The news is the latest blow for Palace, who looked set to appoint Mackay until it emerged he was being investigated by the Football Association over allegations he sent homophobic and sexist text messages while manager at Cardiff.
It is claimed that he and Iain Moody exchanged "disrespectful" messages, and the revelations have led to Moody resigning as Palace's sporting director.
A statement issued by the League Managers' Association revealed that Mackay has denied the allegations, and that he will co-operate fully with the FA probe.
With Sherwood now out of the running to replace Pulis, Neil Lennon is now the favourite with Sky Bet to become the new Palace manager while Steve Clarke and Martin Jol are believed to be among the other candidates in the frame.