O'Kelly was working as No 2 to Dean Smith at the Banks's Stadium but will now resume a partnership with City manager Sean O'Driscoll which yielded promotions at Bournemouth and Doncaster.
O'Driscoll told the club website: "Richard is an excellent coach whose opinion I respect and has the sort of character that every football club needs.
"He will bring enthusiasm and positivity to everything he does and I'm delighted he has joined me as I know it was a tough decision to leave Walsall, where he was achieving great success on a limited budget."
City's football managing director Jon Lansdown said: "We're delighted to welcome Richard to the club as he links up with Sean again.
"They've worked together well in the past, building up clubs and achieving success at the same time."
Walsall chief executive Stefan Gamble said: "Despite our best efforts to keep Richard here, it was evident that the opportunity to work in the Championship and alongside Sean O'Driscoll was too good for Richard to turn down.
"Therefore, after lengthy discussions, it was felt that it was in the best interests of all concerned to allow Richard to take up this new challenge.
"We thank him most sincerely for his services to the club and wish him well in the future.
"The process of finding a successor has now begun and the compensation that we have received will be made available to the manager to fill the position and also strengthen the squad."