In Saturday's League One clash at the Galpharm, Gary Roberts fired in a free-kick from the right byline which Tom Clarke flicked on and Novak pounced to fire a bullet header into the roof of the net from close range.
Striker Theo Robinson had put Huddersfield ahead with a first-half penalty awarded for Paul Reid's 17th minute handball.
It was the former Watford man's 16th goal from as many starts this term.
And it looked as if Huddersfield would have to settle for a 1-1 draw after Colchester winger Simon Hackney levelled in the 77th minute.
Ex-Barnsley striker Kayode Odejayi crossed to find Hackney unmarked at the back post to slot home.
But Novak had the final say with his late winner.