The big spending Canaries had lined up a deal for the Swedish international, but he decided against a move to Carrow Road.
And the 27-year-old says he did not get offered what he wanted, and he will not be forced to leave.
"The clubs were happy with the agreement, but I did not get what I wanted." Toivonen told Expressen.
"I am happy in Holland and the family enjoy themselves and if I don't get what I want, I don't move.
"There are three parties who need to be pleased."
The Sweden international remains uncertain about his future at PSV.
"Either the club do not want or you don't get the conditions you want or the club do not want to sell you." he added.
"I have experienced all of them."