The sides meet for the second time in less than a fortnight at the Stadium of Light on Tuesday evening, live on Sky Sports 1HD, with a semi-final place at stake.
Chelsea arrive on Wearside, where they won 4-3 in the Premier League on December 4, sitting third in the table and only two points behind leaders Arsenal.
Sunderland are rooted at the bottom of the standings and Poyet wants them to learn how to become winners by upsetting the odds and beating high-flying Chelsea.
Poyet said: "If we want to get out of the relegation zone, we need to become winners, winning football games, and to become winners, we need to win the next one, so let's see if we can use it for that reason.
"It's important. It's a cup competition. We are losing a little bit of belief in England in the cups because, of course, the Premier League or the Champions League means everything.
"It's a competition and you cannot start the season thinking, 'I won't care about this one, I will care a little bit about this one and only care properly about this one'.
"It's just about playing and becoming winners. A winner is somebody who wins most of the time, and that's what we are trying to achieve here."
Sunderland face Norwich at home on Saturday but the 46-year-old Uruguayan is adamant that the showdown with the Canaries will not even enter his head until the final whistle sounds on Tuesday night.
Poyet said: "I am trying - it's not easy because of the situation we are in - but I am really trying to concentrate only on tomorrow, so I am not watching Norwich, I am not reading any news about Norwich.
"It's all about Chelsea now. Now, I am sure as soon as the game is finished, it's going to come into my head, Norwich, Norwich, Norwich.
"But for now, I don't want even to think about it."