Church, who signed a two-year deal after being released by former club Reading, struck in each half with two clinical finishes.
Danny Green's long-range free-kick just after half-time and Joe Pigott's injury-time penalty ensured a comfortable passage through to the second round for Chris Powell's men.
Despite Saturday's opening day defeat to newly-promoted Bournemouth in the Championship, Addicks manager Powell opted to change all 10 outfield players, only one of which was enforced due to Bradley Pritchard's suspension.
But they took the lead after 18 minutes when Church struck from 15 yards after strike partner Pigott's goalbound effort was deflected into his path.
Yet Oxford came back into it and James Constable tried his luck from distance but was just off target.
The visitors' best chance fell to Ryan Williams, who was denied from close range by Addicks goalkeeper Ben Hamer after latching on to Johnny Mullins' through ball.
Midfielder Williams wasted another chance to level before the break when he curled over when well-placed inside the area.
Danny Green scored in the 49th minute with a direct free-kick from 25 yards that beat the seemingly unsighted U's goalkeeper Ryan Clarke.
Church scored his second after controlling a 57th-minute pass from Cedric Evina before burying an unstoppable shot past Clarke.
The hosts coasted to victory with Pigott easily converting a spot-kick with the last kick of the match after Jake Wright had tripped substitute Jordan Cousins.