Paul Clayton put the non-league side ahead after just four minutes on Sunday but Orient's Dean Cox levelled on 25 minutes and Dave Mooney scored twice before half-time.
Alfreton pulled one back through Ben Tomlinson and hit the woodwork four times in total before Cox's second killed the game off late on.
Orient boss Russell Slade said: "They started the first half fantastically well, the home team put us under a lot of pressure.
"They got themselves a goal, and to be fair they could have got another one too. We were hopeless in the first 20 minutes.
"We got the goal and that put us in a good sort of rhythm, and then we got two more great goals and that probably has settled the tie.
"We came here and showed them a great deal of respect. It could have finished six five and that is what the cup is all about."