The relegation-haunted Gulls went ahead when on-loan Notts County midfielder Joss Labadie headed them into a second-minute lead before top scorer Rene Howe made it 2-0 six minutes later by deflecting on-loan Sheffield United winger Jordan Chapell's goal-bound shot past Tommy Lee.
Liam Cooper's 77th-minute reply was too late in the game for the Spireites as United held on for an important three points.
Knill said of his first win as Gulls boss: "It was coming. We have drawn the last two and I thought we should have had more, and now we have got that all-important win.
"It was a great start, just what you need and to be fair it was a great goal. Probably when you watch it back you will see that. Great passing, great movement, great delivery and then a great header from Joss.
"So that gets you going and I'm never one for resting on my laurels and sometimes you can score too early.
"Then we score a second and everyone is thinking 'great, it's going to be easy from now on'.
"But you know when you look at the quality in Chesterfield's side it's not going to be easy and you have to dig in and we did that when we had to and right up until the end.
"I am delighted with them, but now we must go again because there are still seven matches to go. The next one here is on Saturday and another big game, York, so we go again."
