Novak blasted into the bottom right corner on 40 minutes after Jordan Rhodes threaded a diagonal pass to him between two defenders.
Rhodes then sealed a first win in four league matches with a glancing header from a Gary Roberts corner in the first half of stoppage time.
Gillingham, who have now lost 11 out of 12 away games this season, made the perfect start as they searched for a first away win of the season.
Simeon Jackson put the Gills in front with an audacious third-minute lob after the home defence lost sight of Chris Palmer's long ball over the top.
Huddersfield struggled to get going against the visitors until Novak made the breakthrough but after that they could have won by more.
Anthony Kay hit the bar with a 30-yard thunderbolt while Novak had a goal-bound header cleared off the line as Town turned the screw.