The Blues were beaten 3-2 by the visitors at Stamford Bridge after seeing Branislav Ivanovic and Fernando Torres sent off in the second half.
Di Matteo was unhappy with Torres' dismissal after Clattenburg adjudged him to have dived and handed the striker a second yellow card, despite Jonny Evans appearing to make contact.
The Chelsea boss also insists Javier Hernandez's winner for the Red Devils was from an offside position.
He told Sky Sports: "The key decisions from the referee today probably influenced the result.
"He sent off Fernando Torres and the third goal is offside and that clearly has decided the game for our opposition tonight.
"He's (Torres) put the ball between his legs and he's through and he gets kicked on his shin and he goes down. It's as simple as that. It's a free-kick for us. It might even be a sending-off for Evans because he's through on goal.
"I was shocked. I thought it was going to be a free-kick for us, and a foul for us, not against us.
He added of Hernandez's goal: "I watched it again and the replay shows when Rafael shoots he's in an offside position."
"Everything went for United in that sense and that really makes us unhappy," said Di Matteo.
"He's (Clattenburg) going to watch it again and probably realise he made mistakes. We can't change the result now and that's the biggest disappointment that we have."
Di Matteo believes Chelsea were still in with a chance of winning the encounter up to Torres' sending off.
He said: "I think we had a slow start and were a bit unlucky with the first goal, it bounced off the post and onto David Luiz.
"The second goal I think we defended badly. But then we had a great comeback and then I thought the momentum had swung for us.
"The most likely team to win the game was probably going to be us. I thought when we went down to 10 men we would have been able to cope but the second sending off made the game really difficult for us."