Spurs narrowly missed out on Champions League qualification last season and one of the most telling blows came in March, when Fulham won 1-0 at White Hart Lane.
Tottenham's season began to unravel soon after as they dropped points against Wigan and Chelsea, and Walker has not forgotten the impact of the loss to the Cottagers.
"Last season when Fulham came to White Hart Lane they beat us, so we owe them something for that," said the Spurs right-back.
"I don't like losing any game, but I think that game was the turning point from last season."
Fulham have lost their last six games, which cost Martin Jol his job, but Walker believes their previous form will count for nothing as the players will be out to impress new boss Rene Meulensteen.
"It will still be difficult," he said. "No game is easy. It's a London derby. They are fighting for things just like we are fighting for things.
"We need to dig in like we did against Manchester United on Sunday, show enthusiasm and desire that we want to go out and win and if we do, then I think we will get the result."