Keith Lowe went close with a header from Matt Richards' corner for the home side in the 10th minute.
Byron Harrison put Cheltenham ahead in the 12th minute, touching in from close range after Troy Brown's header from another Richards corner.
David Noble sent an effort just over the bar from long range but York levelled in the 20th minute.
Ryan Brobbel's corner was nodded in by Bowman for his first goal of the season.
Jamie Cureton headed straight at Michael Ingham from a good position as Cheltenham continued to have the best of it.
And they regained the lead in the final minute of the first half when Jason Taylor was tripped by Tom Platt.
Richards' free-kick curled into the net with Troy Brown initially flagged for being in an offside position. But after consulting with his assistant, referee James Linington allowed the goal to stand.
Cheltenham gifted York a leveller in the 54th minute, though, when Sido Jombati and goalkeeper Scott Brown got in each other's way trying to deal with Luke O'Neill's cross and Bowman nodded into the empty net.