The 19-year-old joined the Cumbrians in August but only recently returned to full training after an injury lay-off.
A mutually agreed recall means James leaves Brunton Park after just one month, having initially signed a deal until January 2014.
However, despite the departure, Carlisle caretaker manager Graham Kavanagh has secured the services of Hull City defender Conor Townsend as cover for the injured Chris Chantler.
"We brought Conor Townsend in from Hull as cover and we are now in a situation where Chris Chantler is well ahead in terms of where we expected him to be with his recovery," he told the club's official website.
He added Manchester United have been "really understanding with us" in regards to the injury sustained by James during the loan spell.
James was only able to make two appearances during his time at Carlisle, one in their penalty shoot-out victory against Blackburn Rovers in the first round of the Capital One Cup and the other in the Blues' season opener with Leyton Orient.
Meanwhile, the club also announced that Chantler has joined the youth team for training this week and could be back with the first team next week.