The Black Cats battled hard at the Stadium of Light but were undone by poor defending as Andy Carroll and Mohamed Diame put the Hammers 2-0 ahead.
Adam Johnson reduced the arrears but Sunderland could not find an equaliser and are now facing an uphill struggle to survive in the Premier League with eight games left to play.
Poyet told Sky Sports: "It's difficult to accept, sometimes when you see a team at the bottom they have no spirit, heart and give up.
"I don't think it's possible to see my team giving up.
"But if you see my team giving up before the end of the season, you won't see me here.
"The downside of the game (is the defending) and we knew that was the way they could hurt us. We will have a few chats about that."
Poyet felt his men had been worthy of a penalty when Kevin Nolan appeared to elbow the ball in the box late in the first half when the score was 1-0.
"It was clear, deliberate. He tried to play the ball with his hand," said Poyet.
"You would need to ask the referee. It was easy to see but that's the way it's been.
"I hope we get some luck; a decision from the ref or a deflected shot goes in.
"It's about winning now. We have a chance and will go to the end."