The 24-year-old Brazilian picked up a groin problem while training in Denver and will fly back to Manchester to continue treatment.
A statement on the club's website read: "As he [Rafael] was unlikely to participate in the remaining tour games, returning to the Aon Training Complex affords the best overall option for his recovery in time to continue pre-season preparation when the team returns."
Louis van Gaal's reign as United manager began solidly in the Colorado capital as they followed up a 7-0 rout of LA Galaxy by recording a 3-2 victory over Serie A giants Roma on Saturday.
But the Dutchman already faces a selection dilemma with the news that Rafael will not feature in any of United's two remaining US tour games, leaving part-time right-backs Phil Jones and Chris Smalling in contention for the role.
Another option is to adopt a 3-5-2 formation, with the versatile Antonio Valencia operating at wing-back - a system Van Gaal used to good effect when guiding Holland to the semi-finals of World Cup 2014.