Robins centre-back Steve Elliott saw a header cleared off the line by Harry Worley in the 23rd minute.
Darryl Duffy opened the scoring for the Robins with a neat finish with the outside of his right foot after Russ Penn's pass in the 31st minute, but Tom Craddock reached a long clearance from Ryan Clarke to level four minutes later.
Keith Lowe restored Cheltenham's advantage in the final minute of the first half, blasting into the top right corner after an indirect free-kick in the box was floated to him by Shaun Harrad.
Worley equalised for Oxford in the 67th minute, leaping highest to nod in Peter Leven's corner at the far post.
James Constable's first goal since February gave the visitors the lead four minutes later as Cheltenham were undone by another ball over the top. And Leven scored from the penalty spot in the 80th minute after Elliott caught Simon Heslop in the box.
