The 22-year-old, who can also play in midfield as well as right-back and centre-back, was out of contract at Huish Park but the Glovers will receive an undisclosed compensation fee from their local rivals due to the defender's age.
"Luke is a great signing for the football club," City boss Steve Cotterill told the club's official website. "He's another exciting young player and we've got him for an excellent fee.
"He was instrumental in the club's promotion from League One and did very well in the Championship last season, winning a raft of awards.
"He's a very versatile player and therefore offers us competition and cover in a variety of positions. We're working towards a smaller squad this season so we need that versatility."
Ayling added: "I spent four years at Yeovil but I think now is the right time for me to move on. I had a great opportunity to come to Bristol so I snapped their hand off really - I just can't wait to start playing and hopefully help take the club into the Championship."
In four seasons with the Glovers Ayling made 185 appearances.