The 23-year-old former England youth international will stay at New York until the end of the season and could make his debut against Crawley on Saturday.
Smallwood, who has made 13 appearances for Boro this season, becomes the fifth January signing at New York following the arrivals of Miles Addison, Wes Thomas, Tom Hitchcock and Blair Turgott.
After a busy three weeks, Rotherham boss Steve Evans admits his work is almost done.
"He is a very, very talented boy," Evans said of his latest addition. "There were four or five clubs that were given permission to speak to him, but he preferred us.
"It probably takes us to a player or two away from getting ready to go forward after the window.
"We haven't stood still, we have made ourselves significantly stronger as a team."
As well as looking to add another couple of recruits, Evans is also working to extend to loan stays of Newcastle pair James Tavernier and Haris Vuckic.
Both men have enjoyed success at New York having signed in November, with Tavernier particularly impressing in both the right and left-back positions.
Their spells have been extended once and Evans is waiting on Newcastle boss Alan Pardew returning from a warm-weather training camp in Dubai to get the deals done.
"Newcastle have been away in Dubai and we have not been able to speak to the powers that be," he added.
"Both players have said very clearly that if they could stay then they want to. We would love them to stay here until the end of the season.
"Only Alan Pardew can make that decision and I am sure he will give it due consideration. We are hopeful that Alan will say yes.
"That might be Monday, but when Alan is ready to give us that decision he will.
"James needed the opportunity to come and play games and he has done that and become a fans' favourite.
"Haris is still working his way back from injury and he is very focused and knows what he wants."
Evans also revealed there are not expected to be any outgoings unless the right offer comes in.