Chambers was originally scheduled to stay with the Daggers until Saturday, but the U's requested the player returned early in order to assist their promotion push from the Conference.
The winger made six appearances for Wayne Burnett's team and created an assist on his debut against Exeter last month.
Burnett said it was "disappointing" Chambers' loan was cut short.
"We wanted to try and extend it for a bit longer, but they're within their rights to call him back and that's what they've done," he told the club's official website.
"He will be a loss for the team, he's fitted in well with the squad, but we have players with the ability who can come in."