Villa are currently in the bottom three of the Barclays Premier League ahead of Saturday evening's derby clash with West Brom at the Hawthorns.
Lambert has only limited funds to spend compared to the combined £30million club owner Randy Lerner spent on Darren Bent and Jean Makoun two years ago.
But the Villa manager is adamant any newcomers will have to improve the current squad.
Lambert said: "We'd try to do something if I thought the people we could get in would make it better.
"But, if we can't do it and I have to go with the group that's there, then I'll go with what's there.
"What I don't want to do is fill the place with people I'm bringing in just for the sake of bringing them in.
"It's the wrong thing to do for this club. It's wrong if they're no better than the ones who are already here.
"I have to bring in people I think will make a major impact."
When asked if only has £3million to spend, Lambert said: "I know what's happening so I don't have a problem with that.
"If I have to go with the lads I've got, that's who I'll go with. Money dictates a lot. It's tough because of the way the market is."
Lambert is unfazed about the size of the task facing him to restore Villa to former glories after three successive relegation battles.
He said: "I knew it needed rebuilding which was the great challenge. It's not something where I'm sitting and thinking 'I can't believe the way it is'.
"I knew the parameters and what you have to go through to get what you want. I understand that really well.
"It's a challenge and I love the challenge. I'm not going to wilt or feel terrible on a Saturday night.
"I love it, love the challenge of it. I love the pressure of it, and it's something you're determined to get right because, if you do, you know this is an unbelievable club.
"It's always been a big club. But I've been brought in to try and rebuild it and get results while doing it - and it does take time.
"I think someone said the other week it might take you three to four years to do it. Whether I get that time.....I understand what they're saying."