The Norwich loanee, who previously had two spells on loan at Crystal Palace, netted midway through the second half to give Mark Lillis and Steve Eyre a victory in their first league game in joint-caretaker charge and Town's first win in 13 Championship matches.
It means the Terriers, who parted company with Simon Grayson a few days after the 4-0 loss to Watford, move up to 16th and eight points clear of the relegation zone. Palace missed out on the opportunity to climb into second place and they now have just one win from their last nine league matches.
Scott Arfield tried his luck from distance early on but Julian Speroni comfortably held on, while Yannick Bolasie tested Alex Smithies at the other end.
Neil Danns forced a smart save from Speroni midway through the half and Oliver Norwood's subsequent corner flashed across goal, with Vaughan inches away from making contact.
But the Eagles did create some pressure of their own, most notably through the lively Bolasie, who linked well with new Manchester United signing Wilfried Zaha, only for Calum Woods to intervene before either could get a shot away.
There was a half-chance for Glenn Murray just before half-time, but after latching on to Andre Moritz's ball over the top, the former Brighton striker's effort cleared the crossbar.
The visitors' resurgence continued after the interval, with substitute Jonathan Williams and Bolasie breaking clear in separate attacks only to spurn the opportunity to test Smithies.
Jacob Butterfield almost marked his 100th career league appearance with a goal, but the Norwich loanee fired narrowly over the bar from the edge of the box after sloppy play from Jack Hunt.
Another on-loan Canaries man proved more decisive for the hosts with 65 minutes gone. Arfield's superb pass released Vaughan and he kept his cool to slot in his fifth of the season.
Bolasie almost levelled within five minutes but his free-kick whistled past Smithies' upright.
Holloway introduced Jermaine Easter for the last 15 minutes and switched to an ultra-attacking formation as he sought to inspire a late turnaround. The Welsh striker's first involvement saw him flick a header just wide of the far post.