Sunday's 2-2 draw with Everton marked Spurs' failure to win for the third home game in succession as they chase Europa League glory and qualification for next season's UEFA Champions League.
And the anxiety among supporters who witnessed a late-season collapse last year has not gone unnoticed by the players, according to Dempsey, who expects more of the same when Manchester City visit later this month.
He said: "That's the way it's kind of been the last few games at home. It's been tense. I've felt like that's how the atmosphere has been.
"I don't know if it has been affecting performances. It's been noticeable, in the last three home games. There are a few teams right there in the fight for the Champions League and it makes it more interesting for people to watch.
"It's a little bit more stressful but it's a good stress. At least we're pushing to try to get that spot.
"It's going to come to the end and, hopefully, we have enough character to get the job done. You've got to keep going and do everything you can to help the team.
Spurs go to Basel this week with their Europa League quarter-final tied at 2-2 from the first leg, while in the league Arsenal are now guaranteed a top-four finish at their rivals' expense if they win their remaining games.
Dempsey said: "The race for a top-four finish will be a fight all the way until the end. We would have liked all three points against Everton but it was an important point."
Meanwhile, Younes Kaboul looks set to make his long-awaited return from injury in Tottenham's Under-21 match against West Ham on Monday night.
Kaboul has not played since the opening day of the season when he aggravated a knee injury which had also kept him out of the European Championship.
The 27-year-old, who was Tottenham's most consistent centre-back last season, underwent surgery and was initially expected to be out for four months, but he has struggled in his rehabilitation from the injury.
The Frenchman will get a chance to prove his fitness, however, as he is understood to be in the Under-21 squad that will face the Hammers at Upton Park.
Should he come through that match, the former Portsmouth man could be in contention for Tottenham's away trip to Basle on Thursday, although a more realistic comeback game would be the club's game against Manchester City on April 21.