The Terriers, who lost 5-0 to Southampton in League One in midweek, got off to a nightmare start when they conceded after seven minutes before Nathan Eccleston was shown a straight red on 26 minutes. And the Gills capitalised as they found a second goal for the first time in 2010 thanks to Jackson's 17th strike of the season.
Rene Howe had already gone close for Gillingham before they did take the lead when Jackson lashed home from the corner of the box beating Alex Smithies at his near post.
The visitors' hopes took a further blow when Eccleston was giving his marching orders for a horrendous, two-footed challenge on Stuart Lewis.
The Gills took advantage of the extra man within seconds of the restart as Howe's flick put Jackson clear and the Canadian international finished emphatically.
But Huddersfield did come into the game as Theo Robinson had a goal disallowed and fellow sub Gary Roberts hit the post while Jackson missed two chances for his hat-trick.