Williams impressed Mick McCarthy when he scored for Palace's Under-21s in their 2-1 win over Ipswich at Portman Road on Monday night.
The 20-year-old Wales international has yet to make a Premier League start for Palace this season and is joining Ipswich to play more first-team football.
Palace chairman Steve Parish told the London Evening Standard: "Jonny is going for a month initially because he is very close to our first team.
"He just needs to get a bit more sharpness. If you look at people like Wilfried Zaha and Tom Ince, they have played 100 games in the Championship.
"Jonny has had a very stop-start last couple of years and needs games. He played very well on Monday. He just needs to play week in and week out.
"We didn't want to let him go until the end of the season."
Millwall are also hoping to complete the loan signing of Palace midfielder Owen Garvan before Saturday's home game against Brighton.
The two clubs are close to finalising a deal that will run until the end of the campaign after Garvan turned down a move to Sheffield Wednesday.