Two wonder strikes from Pilkington propelled the Terriers past the second round stage of the competition for the first time in eight years.
Town were already leading 1-0 thanks to a goalkeeping blunder when Pilkington smashed in the first of two superb efforts into the top-left corner in the 12th minute.
A long headed clearance out of defence by Peter Clarke was turned into Pilkington's path by Lee Novak and the ex-Stockport winger drilled home from 20 yards. He repeated the trick in the 24th minute after a solo run past three defenders, with keeper Joe Lewis given no chance with a finish which curled wickedly into the top corner.
Earlier, Lewis had been at fault in gifting Huddersfield the lead by allowing a Graham Carey shot from distance to slip through his legs.
Posh, who were woeful in the first half, made three substitutions at half-time and came close to a remarkable comeback after the break.
Substitute Aaron Mclean headed in off a post from a cross by fellow sub Lee Tomlin after 50 minutes.
Peterborough then pulled another one back through Mark Little's close-range header three minutes from time, but Huddersfield held on.