The Crystal Palace midfielder suffered a hip injury in his side's draw with Swansea at the weekend.
He is the second player to pull out of the clash at the Cardiff City Stadium after Charlton forward Simon Church.
"Joe Ledley will be returning to Palace," confirmed manager Chris Coleman.
"He has got a hip problem that never really settled down after Sunday's game. We won't take the chance with Joe."
Wales were already without two other midfielders in Arsenal's Aaron Ramsey and David Vaughan, who is currently on loan at Nottingham Forest from Sunderland, due to injury.
On a more positive note, captain Ashley Williams has declared himself fit after a sickness bug curtailed his involvement for Swansea at the weekend.
The defender was sick on the field during the 1-1 draw with Palace before leaving the action after 70 minutes.
Williams said: "I feel better now. I don't know what it was. A few of the boys at Swansea have been ill.
"Saturday night and Sunday were not great for me but the last couple of days I have been okay.
"I have been eating well and resting, so I'm okay."
Star player Gareth Bale has reported fit from Real Madrid and is set to play a full part in the game.
Bale has hit form for the Spanish giants in recent weeks having scored five goals in his last seven games.
Coleman said: "I was never worried about 'Baley'. Here has been here before when he's known he is not going to play and has made the trip anyway to be with us. It is great to have him.
"He was involved in a tough game at the weekend - the Madrid derby was brutal at times.
"My heart was in my mouth on one or two occasions but he has come through that. He is here and raring to go."