Arnason tucked in a rebound on 44 minutes after his shot from a Jack Hunt cross came back off a home defender.
The Swedish-born Iceland international, a key figure in only a third away victory all season, then helped create his side’s second for Conor Sammon.
He burst forward on 60 minutes and played in Lee Frecklington, who teed up Sammon for a 20-yard low drive.
The relegation-troubled side, fourth bottom before play, had not won at Huddersfield since 1964.
Rotherham made most of the early running, with a Matt Derbyshire shot from a Sammon cross cleared on three minutes.
On-loan Derby striker Sammon shot at Smithies when left unmarked from a Ben Pringle cross five minutes later.
A couple of surging runs from Joe Lolley, making his first home start, were all Town offered early in the game.
Lolley also forced Millers keeper Adam Collin to save a long-range free-kick on 26 minutes after he was fouled.
Arnason was denied a goal on 40 minutes when his header from Pringle’s cross was tipped over by Town keeper Alex Smithies.
But soon after former Terriers defender Hunt skinned Jack Robinson on a trademark run down the right.
Hunt’s cross was stabbed at goal by Arnason at the far post and when the effort was blocked, he was first to react to the rebound and put Rotherham ahead at the break.
Just before Rotherham’s second-half goal James Vaughan drew a good save from Collin with a stinging 20-yard drive.
Frecklington was sent off on 74 minutes for a foul on Conor Coady, having been booked in the first half for a lunge on Joel Lynch.
But even with 10 men, Rotherham looked more dangerous and could have won by more.
A weak back pass by Robinson let Sammon in on 81 minutes and Smithies had to race out of his box to clear.
One of the few chances for Huddersfield late on saw Collin save a shot from substitute Radoslaw Majewski.
Rotherham kept Town pinned back and won a series of corners to help see the game out.
Huddersfield have now lost three matches in a row, having conceded nine goals and scored only one. The final whistle was greeted by a chorus of boos from home fans in a crowd of 13,342.
Meanwhile, United boss Steve Evans, known for his theatrics, raced to the away end to give a clench-fisted salute to their supporters.