Lambert found himself targeted again on Saturday as Villa slumped to a demoralising 4-1 defeat away at Swansea City.
A disgruntled fan base has started to call for the Scot's head on the back of a seven-match winless run which has delivered six defeats.
An untimely wobble means Villa are still looking anxiously over their shoulder, with only a three-point cushion keeping them clear of the relegation zone.
On-loan striker Holt admits that the situation is far from ideal and that performances have dipped, but he wants those on the pitch to shoulder responsibility rather than those in the dugout.
He said: "I think it's fair to give the players a bit of stick.
"The manager has put a team out there to perform and we have not got the points.
"As a group of lads we have got to take the flak and stick for that as we have been beaten 4-1 away from home.
"We are not in a good vein of form at the moment and we are giving away cheap goals.
"It's always the way when things are going against you that people look at the manager. But we deserve the stick.
"The belief among the players is still there. We have already had a chat and we just have to put this one to bed.
"We have got three games left and they are all big ones. But it's still in our hands."