Scot Ferry made 22 appearances for Pompey since arriving from Swindon last summer but the former Celtic man, whose campaign was hampered by an ongoing back injury, is now keen to head back north.
Goalkeeper Sullivan played seven times during his year with the Fratton Park side and spent a spell on loan at Cambridge.
Manager Andy Awford told the club's official website: "Unfortunately, Simon and his family have not been able to settle in the south and for personal reasons he has decided to move back up north.
"Hopefully he will be able to return to full fitness and go on to have a successful career.
"It was a mutual agreement with both players and we wish them well for the future."