The Shots, who were booed off by their own supporters at the weekend, got off to the worst possible start as Paris Cowan-Hall opened the scoring after just 34 seconds at Home Park on Tuesday night.
Craig Reid equalised with his seventh goal of the season but Argentine midfielder Andres Gurrieri scored a superb 30th-minute winner to seal a 2-1 win for the Pilgrims.
Holdsworth said: "We scored a fantastic goal but gave them two goals. It was catastrophic defending for the first goal in the first minute of the game.
"We have had two meetings in the last two days and we spoke about doing the right things and we have given them a goal. We saw it coming and didn't deal with it. And for the second goal, again we have given them too much space to run into inside and he came off the man and scored a decent goal. But we could have done something about it.
"I am disappointed with the defenders and the defending side of the game, we are gifting players goals at the moment. We seem to get punished every time a team attacks against us. There is vulnerability there and we are having that sort of run of games where we certainly need to be tighter at the back."
