Reading took a two-goal lead in the first half when Simon Cox scored from long distance and Nick Blackman neatly converted a penalty.
Millwall deservedly dragged themselves back into it with efforts from Ricardo Fuller and Mark Beevers and looked the better team towards the end but – Reading struck with six minutes remaining when Cox stooped low to nod in a Jordan Obita corner.
After four successive wins in all competitions, without conceding a goal, Reading began with confidence.
However, they were indebted to goalkeeper Adam Federici for tipping over a firm header from Fuller early on.
They took the lead in only the sixth minute. Federici launched a huge clearance downfield and Cox unleashed a 20-yard drive past Millwall goalkeeper David Forde for his third goal of the season since rejoining the Berkshire club from Nottingham Forest.
Millwall, with only one win in their previous five matches, struggled to make any significant headway.
They could have gone further behind soon after, when Glenn Murray nodded over a free header from an inviting Obita cross.
In the 15th minute, Millwall did fall 2-0 behind. Blackman drove in from the left flank and was brought down by Byron Webster. Blackman took the penalty himself and comfortably despatched it past Forde for his second goal in as many games.
Millwall woke up and dominated the rest of the half, with Federici making a fine double save to deny both Scott Malone and Scott McDonald.
However, the Australian could do little when Millwall reduced the deficit in the 39th minute. Fuller, a danger throughout the opening half, shimmied in the Reading area and fired a low shot past Federici’s despairing dive.
Millwall manager Ian Holloway had to be spoken to by referee Keith Hill as his frustrations boiled over on the touchline – but he was let off with a warning.
Reading attempted to reassert their authority early in the second half, with Murray going close to connecting with a Jamie Mackie cross.
But Millwall were level in the 54th minute after a period of sustained pressure had tested the home defence.
Substitute Martyn Woolford slung over a corner from the right, Federici missed it and defender Beevers headed home easily at the far post.
The sides then cancelled each other out but Reading made the decisive breakthrough when Obita’s corner was nodded in from close range by Cox.