Clayton, 23, had played a key role for Leeds last season and had not expected to be sacrificed by Warnock, who was desperate to free-up finances at Elland Road in a bid to strengthen his squad during the summer.
"I went into (Warnock's) office expecting the end-of-season chat and he said 'I'm selling you,"' said Clayton, for whom former Leeds boss Simon Grayson paid £400,000 to take him to Huddersfield.
"It was quite a shock to the system and I was quite upset on the way home. I didn't think I was going to go.
"I'd been looking at houses in Leeds and it was a case of walking into the meeting laughing and discussing what everyone else was discussing - which was 'who do you think are the players he's letting go?' I wasn't expecting the old hammer blow.
"He said he was more than happy to keep me, but he got an attractive enough offer to let me go. I decided to leave and I'm very thankful to have come to a place on the up."
The former Manchester City trainee, who spent two years at Leeds, added: "It was nothing to do with my terms or anything like that. The gaffer agreed to what I wanted and he went in to try and do it. It broke down a bit higher up I think and we had to part ways in the end."