The 50-year-old, who has been heading up the Robins' academy since October, will also sit on the board of directors with current CEO Doug Harman remaining as a director after Pelling's appointment.
Pelling, who spent 14 years as Nottingham Forest's finance director, told the club's official website: "I am delighted to be joining the board of Bristol City as chief executive. It is an exciting time with the stadium redevelopment project due to provide the club and its supporters with the facilities that they deserve.
"My seven months at the academy have given me a great introduction to the club. I feel that the club provides a great opportunity to give promising players a platform to develop their talents.
"I now look forward to working with club staff as we move towards a bright future."
Vice-chairman Jon Lansdown added: "John boasts more than 15 years of Board-level experience principally with Nottingham Forest.
"His time in charge of the academy has been very successful and we're delighted to welcome him on to the board from June 1."