The Scot is under intense pressure following a poor run of form that has seen Villa lose five of their last seven games in all competitions.
The midlands side's home record is also disappointing, with just two victories and six defeats from 10 top-flight games.
Lambert's side were also knocked out of the FA Cup by Sheffield United last weekend and they face a tough task in the Sky Live game against Arsenal at Villa Park on Monday night.
Lambert says that he has a good relationship with chairman Randy Lerner, but always works with "that fear" of losing his job.
He said: "I never stop worrying. I'm not immune from it.
"The relationship between me and Randy is what it is. You are never immune from anything, and I would never view it that way.
"You just keep going - there is no secret formula to it. You just keep going and don't lose focus on what you are trying to do. I will never waver from that."
And asked ahead of the Arsenal contest how he had been feeling in the days since the cup tie, Lambert said: "The same way I do after every other game (Villa have lost).
"I don't sleep great, I don't eat great. The usual stuff.
"It is just the life of a football manager. It is a great job, but you don't feel great (after defeats).
"And contrary to other people's reports, I never degraded the competition at all.
"I tried to win the game, I put the best team out I possibly could and we got knocked out. Good luck to Sheffield United. We weren't good enough."