The Robins went into the game with only one league win to their name this term but a first-half strike from Danny Haynes was enough to see them home.
They had David James to thank in part, particularly for a fantastic second-half stop to deny Royals frontman Noel Hunt.
Cole Skuse was ruled out for the hosts with a heel problem after limping out of the weekend defeat at Cardiff so Lee Johnson slotted into the centre of midfield alongside Marvin Elliott.
Jamal Campbell-Ryce was preferred to Danny Rose on the wing as Millen looked for a first home win of the season and only a second of the campaign so far.
The visitors, looking to move into the play-off places with a victory, brought in Hunt for his first start in over a year with Hal Robson-Kanu dropping to the bench.
The Royals travelled in confidence after losing just one of their five away league games this term and Shane Long showed good pace early on but his ball for Hunt, when in a bit of space, was poor.
City almost went ahead in the third minute when Albert Adomah’s solo run and cross found Jon Stead but his powerful strike came back off the underside of the bar with Reading goalkeeper Adam Federici beaten.
Stead went close again 10 minutes later with Stead again a menace, turning inside Ian Harte before forcing Federici to tip the ball round his post.
Both sides looked up for the fight but it was City who edged ahead in the 28th minute.
A ball to the far post was headed back across goal by Adomah where Stead saw an ambitious overhead kick blocked by Matt Mills, but the ball fell kindly for Haynes to hammer home a half-volley for his first goal of the season.
Long tried his luck with a fine effort from range as the Royals looked for a way back into the game but his effort was too high while Jimmy Kebe forced City keeper James to parry before the break after a slick attack from the away side.
James was called upon early in the second half and showed the class that made him England’s number one.
The Royals strung together a series of passes before Andy Griffin sent over a great cross for Hunt, who headed goalwards, but James produced a fantastic stop to tip the ball to safety.
Adomah was centimetres away from levelling and Reading boss Brian McDermott made his first change by bringing on Brian Howard for Harte on the hour.
James saved from Karacan as McDermott’s men started to turn the screw but Steve Caulker had a chance at the far post for the Robins but headed wide at the far post.
Robson-Kanu and Simon Church had been thrown into the fray in the hunt for the leveller while Millen shored up his midfield by throwing Kalifa Cisse into the fray, Rose also replacing Campbell-Ryce.
Reading continued to press, Jobi McAnuff’s effort being blocked, while he could only send a free header straight at James.
Karacan hurled another long-range effort at James in added time but City held on.