The damaging defeat leaves United 14 points behind leaders Arsenal and Vidic believes the gap is too big for the current champions to close.
Vidic told Sky Sports: "We are too far behind and we have now to focus on the top three or four.
"We have to be positive, today we didn't play a bad game. They had three shots and scored three goals and that's the difference in the game, we have been punished for every mistake we made.
"It's not going to be easy, we have to now fight to achieve the Champions League. We have to stay positive and stick together and work hard. With a bit of luck hopefully we can achieve that."
Vidic is refusing to blame David Moyes for United's slump in form and says the players are tired of discussing their under-fire boss.
He said: "We cannot always speak about managers, the players are tired of speaking about that because the manager is trying to do the best he can. Tactically we did well today."
The Serbian defender was upset by the decision to send him off for a foul on Eden Hazard late in the game at Stamford Bridge. The red card means Vidic is ruled out out of Wednesday's Capital One Cup semi-final second leg against Sunderland.
"My feeling is that it was very harsh because I went for the ball. Hazard we know is a sharp player and he just pushed the ball," he added.
"In my opinion it was not a red card I was disappointed with that decision at the time."
Despite their struggles, Moyes insists that their long-term planning will not be replaced by short-term thinking, no matter what their results are.
Asked if he could change his plans to fix their short-term issues, he told Sky Sports: "Not because of results.
"The club know exactly what we want to do and the direction we want to go and the type of players we want to bring to the club and a lot aren't available in this window. If we could get any of them we would do, we are trying but more than likely most people know it is difficult.
"I think the club have always said it is long-term, we know there is a job to be done and we have an idea of players we would like to bring in and we would like them now, but more than likely won't be."
Moyes refused to rule United out of the title race, but admits it would not be easy from here.
"It is a long way away, we will never say it is gone and we will do our best to get back in and the only thing we can do is try and win our next game and we will try and do that," he said.