Millen, who played for the club as a teenager, admits it was a dream come true to take charge of Palace last weekend but accepts he is not the right man for the struggling Premier League club.
The Croydon-born assistant manager was thrust into the Eagles hot-seat for the meeting with Arsenal following Ian Holloway's departure last week.
Palace impressed despite the match ending 2-0 to the 10-man league leaders and Millen will again take the reins for the trip to West Brom at the weekend.
However, Saturday's match may well prove to be the former Bristol City manager's last in charge with Palace co-chairman Steve Parish hoping to appoint a new boss by the start of next week.
Tony Pulis, Chris Coleman and Martin O'Neill are the bookmakers' favourites for a job which Millen will not be applying for.
"Probably the type of manager that needs to come in to look at this group from the outside would be a fresh view on what is needed, maybe some different ideas," said Millen.
"I have worked closely with the players now for a long time and I enjoy it, but I think someone from the outside coming in will give us a little bit more of a lift. It is right that I don't put myself forward for the job.
"I loved it, I really did and I have missed it, but I enjoy coaching the players as well.
"To be in charge of Crystal Palace in the Premier League was a dream come true.
"But when the game gets going, you want to win the game and I wasn't on the touchline there with a smile on my face thinking 'this is enjoyable' because we were losing after 47 minutes.
"I honestly felt when it was 1-0 that we could get something from the game, even before the sending off - that we enough chances in the game to be at least level and the players deserved that.
"I enjoyed the whole experience, but I was disappointed we didn't get something from the game because we deserved it."