Manager Andre Villas-Boas criticised Spurs fans for not getting behind their players during the 1-0 win over Hull City in the Premier League on Sunday.
Villas-Boas said both he and the players were unhappy about the 'negative' atmosphere which he thinks made it feel like Spurs were the away team at White Hart Lane.
And England winger Townsend has backed his boss by insisting it is difficult for Spurs to break down teams that are only focused on defending.
"It's difficult and sometimes it will take 70, 80 minutes to get the goal," Townsend said in the Daily Star.
"It was difficult (on Sunday) against a team that sets up with five defenders and four midfielders, all behind the ball.
"We tried to play the football, the opposition wouldn't let us with their system and tactics, but our persistence paid off and we got the win, that's all that matters."