O'Brien joins the Sky Blues on a free transfer after leaving Barnsley in the summer and he has signed a two-year deal at the club.
City manager Steven Pressley told the club's website: "We're delighted to bring a player of Jim's quality to the football club.
"He is a player of proven Championship quality and his energy, work rate and team ethic along with that quality makes him a perfect fit for us."The 26-year-old midfielder made 32 appearances for the Tykes last season, netting on three occasions, and he knows Steven Pressley from Celtic, where they were team-mates for a season after O'Brien came through their academy.
Pressley added: "Coming into the summer, I knew we needed to add some quality to our midfield and options on the wing. Jim ticks all the boxes.
"After speaking with him, it was evident to me that Jim has the right mentality to fit into our system and I believe that this football club can provide him with the challenge he wants."