The Robins missed the chance to move into the automatic promotion places with the defeat away at struggling Argyle and Yates was seen to exchange words with an irate fan after the final whistle.
In an official statement on the club's website, Yates said: "I want to apologise for the use of bad language aimed at a supporter.
"My use of language was of course unacceptable and for anyone that I have offended, I am sorry.
"I took my players over to our fans at the end of the game to thank them for travelling so far and supporting us, as I do every away game.
"I saw one supporter approach one of my players and swear and shout at him in a very aggressive manner. The player in question had only come on as a sub with the score already at 2-0 and I am sorry to say I reacted to this supporter.
"We are a close-knit team and I always back my players but I certainly regret what I did and apologise for any offence caused by my language.
"It was a frustrating day for myself, the players and of course the supporters. It is now important to put this behind us with three big games approaching, starting with Gillingham next weekend."