Frank Lampard’s side have now lost to United on three occasions this season, and the latest sees them now just one point clear of the chasing pack in fourth.
And Kovacic admits that Chelsea need to raise their game after a frustrating evening at Stamford Bridge.
He said: “We need to improve a lot because we have tough games ahead of us and we have to be focused.
“We have to train hard every day, to prepare our games better and try to avoid our mistakes because we are making too many mistakes in these kind of matches and you get punished.
“We have a good week ahead of us to prepare for Tottenham and to play a good match. We know this match means a lot for us because Tottenham are close behind us so we need to prepare ourselves quite a lot to win this game at home.
“We are still fourth so we are still in a good position but we need to wake up because they are coming from behind. We need to play these next games like we were playing in the beginning of the season and change some things as well, obviously.
“We played well in the beginning but it’s always the same against United – we play good, we create chances, we don’t score and then they score from one opportunity.
“We don’t have the three points and that’s the most important thing. It was a good game from us, we played well, like we have for more or less the whole season, but we made some silly mistakes and we lost the points.
“We are still a very young team and we need to improve a lot. Now we will see if we are a great team because we have some tough matches to fight for the fourth position and we need to show ourselves.”
Chelsea have a derby date with Tottenham on Saturday, before attention then turns to the first leg of their Champions League last-16 encounter with Bayern Munich.