Huddersfield, who went into the game on the back of three straight defeats, survived a scare after just six seconds when Neal Trotman needlessly conceded a corner but were well worth their first-half lead.
Home keeper Glenn Morris foiled Robinson when he dived at his feet, and also held a shot by Gary Roberts.
Huddersfield went ahead in the 26th minute when Gary Roberts, out on the left, found Rhodes who steered his shot into the far corner.
Jason Demetriou and Jonathan Tehoue could make nothing of rare chances that fell Orient's way.
Huddersfield went further ahead after 68 minutes when Robinson volleyed a cross by midfielder Anthony Pilkington into the roof of the net.
Orient, who slipped to their first home defeat of 2010, did not manage a shot on Huddersfield's goal until added time when keeper Alex Smithies turned a free-kick over his crossbar from Sean Thornton.