First-half goals from by Andy Robinson and substitute Luciano Becchio put City on the back foot but Barry Conlon pulled a goal back in the 71st minute as the League Two side put up a spirited struggle in front of 20,128 fans at Elland Road.
Leeds midfielder David Prutton shot against a post after just four minutes but the breakthrough came when City midfielder Paul McLaren handled and Robinson converted the penalty after eight minutes.
City had a 21st-minute goal controversially disallowed after Joe Colbeck's low shot cannoned off Frazer Richardson and goalkeeper David Lucas in a congested goalmouth.
Becchio replaced the limping Enoch Showunmi in the 29th minute and 12 minutes later the striker headed home a corner from Robert Snodgrass after Robinson had struck the bar from 25 yards.
Conlon punished a mistake by Paul Huntington by drilling the ball in from an angle to give City brief hope but Leeds held on to reach the second round.