However, Yates was also critical of Taylor's reaction to the challenge that saw Oxford's Jon Meades sent off midway through the second half.
"It was a naughty tackle but I don't want Tails reacting like that and giving the opposition bench the opportunity to get at him," Yates said. "I understand why he reacted but I'd rather he didn't.
"He was brilliant with the ball and played some real telling through balls. He has been excellent since he came back in pre-season.
"This is the Jason Taylor we signed from Rotherham. Last year was difficult for us all and Jason just got singled out possibly by myself and sections of the crowd.
"But when Tails does what he does, his attitude is second to none and his attitude in the changing room is a really positive one."
John Marquis gave Cheltenham the lead on the stroke of half-time when he headed in a Joe Hanks corner. Koby Arthur wrapped up the win 12 minutes from time with his fourth goal of the campaign.
"We always talk about goals changing games but that one really settled us down," Yates added. "It was a terrific goal from a well-worked set-piece.
"Koby has then scored a back-post header. I changed my front two and they scored so maybe I've got it wrong.
"I thought we thoroughly deserved to win. We had lots of shots, a couple of goals and another clean sheet so it's a pleasing result."
Oxford's best chance of the game came early on when home goalkeeper Trevor Carson needed to push Joe Riley's 25-yard free-kick around the foot of the post.
U's manager Michael Appleton has yet to see his team win this season, he said: "I think we got caught up getting involved in a couple of things which probably cost us in the end. We lost our concentration a little.
"I have said to the players that they have to really dig deep from the pit of their stomachs when they are having a poor run.
"We created loads of opportunities and I wasn't worried when we went down to 10 men because I thought we would score goals. The difference when you are down to 10 men is that you have to move the ball a little bit quicker and move it a little bit shorter and by keeping the two up there we tried to get it to the front men a little too often."
Appleton did not think Meades should have been sent off.
He added: "It's not a red card. He's a young player who was over enthusiastic and he had to follow through because he slipped.
"It looked a lot worse than it is and to be fair the player couldn't get up quick enough so it couldn't have been that bad. He's a young player and he has to learn from it."