The Blues slumped to a shock defeat at the Hawthorns as their indifferent run of form in the Premier League continued.
Chelsea have now conceded 18 goals in their last eight games in all competitions and collected two points from their last four Premier League matches.
Di Matteo has challenged Chelsea to show they can cope without skipper and defensive lynchpin Terry and stop leaking so many sloppy goals.
The absence of Terry with a knee injury for three weeks after only just returning from a four-game domestic ban has added to their woes ahead of Tuesday's away UEFA Champions League clash with Juventus.
"Clean sheets certainly give you a chance to win games and we certainly have to improve on that side," said Di Matteo.
"Defensively we just weren't good enough. Of course we miss John (Terry). He is an important player for us.
"He gives us a lot of confidence at the back and in general to the team.
"It's a shame that he is injured - but we will have to do without him until he comes back - and improve."