Collins, an academy graduate of the Terriers youth system, latched on to Danny Cadamarteri's pass before sending Peterborough goalkeeper Joe Lewis the wrong way in the 34th minute.
Peterborough almost broke the deadlock after 17 minutes when the lively George Boyd found Chris Whelpdale unmarked in the penalty area, but his close-range drive was parried by Huddersfield's Alex Smithies.
Moments later Phil Jevons flashed a free-kick off target before Collins opened the scoring. But Posh rallied after the interval, with substitute Shaun Batt dragging a shot wide when well placed.
A minute later, Darren Ferguson's Peterborough side should have equalised, as Boyd freed top scorer Craig Mackail-Smith, who had his shot blocked by Smithies.
The hosts were hanging on to their slender advantage, and Peterborough's Aaron Mclean had a rasping shot on the turn superbly tipped over the cross bar by Smithies as they hung on.