Spurs lost 1-0 at home to Newcastle on Sunday despite dominating for large periods of the match.
After investing heavily on new talent during the summer the expectation levels at White Hart Lane were increased but the fans have now seen Spurs lose two of their last three Barclays Premier League home games.
Sunday's defeat at the hands of Newcastle meant Villas-Boas' side missed the chance to move up to third in the table but Townsend is not about to start panicking.
When asked if a lack of goals was starting to concern the players, the winger replied: "If we are winning 1-0 then we will take that because it is three points but if we are losing it is a problem.
"We are a relatively new side, we are a young side and we are still getting to know each other and learn each other's games. Goals will come I'm sure of it - hopefully sooner rather than later."
On a weekend where both Manchester City and Arsenal lost and Everton and Chelsea could only draw it was an opportunity for Spurs to steal a march on some of their close rivals.
But Townsend insists the time to worry about how other teams are faring comes much later in the campaign.
"I think it is a bit too early in the season to be focusing on other results," he said.
"We have just got to focus on our own results and in January, February, March we will see where it takes us and start looking at other results but at the moment it is all about what we do."
Loic Remy's goal condemned Spurs to a defeat ahead of an international break for the third time this season.
Townsend will meet up with his England team-mates in the coming days ahead of Wembley friendlies against Chile and Germany as the 22-year-old looks to build on his promising start at international level.
"I had a fantastic first two caps for England," he added.
"Now it is a chance for me to go away again and prove myself in front of the manager.
"There aren't too many opportunities now to play for England between now and when her picks the squad so I have two games where I have to force my way on to the plane and that is what I will look to do whenever I am given the chance."