City pressed for an equaliser for large parts of the second-half, but were unable to find a way past the hosts' stern defence as Phil Bardsley's 21st-minute strike proved to be the difference.
The defeat was City's third loss from their last four league games on the road, although Pellegrini believed that his side should have come away with at least a point.
"I really cannot believe that we lost this match," he told Sky Sports.
"I think that Sunderland won the match without the possession in the 90 minutes, but football has these things.
"They scored a goal because our defenders had a gap between them and it was the only chance they had to score.
"I think we had several chances at least to draw or to win but we couldn't do it.
"Maybe we tried to play too quickly, but I think that the changes worked. We are playing well but points say that we just have four points away this season - that's not many points."