The Hammers defeated the Blues 3-1 at Upton Park back in December and will be bidding to follow that up at Stamford Bridge, where Allardyce reckons the animosity towards Chelsea manager Rafa Benitez can work in his side's favour.
"You can see the importance of your fans being behind you rather than not," he said.
"I think Chelsea's team has suffered at Stamford Bridge because the fans have not been fully supportive of what has happened. That is nothing to do with Rafa, he can't help that.
"He is a manager trying to do a job the best he possibly can and he's obviously got a great CV at the jobs he has done at his other clubs.
"But it shows the importance of a volatile fan base that turned the other way, what effect it can have on a team.
"In the reverse of that we all know the extra man that fans can create. You see the opposite of that at Chelsea and certainly in the early stages when Rafa took over and Chelsea suffered some poor results because of that. That's the effect the fans have on the team that plays on the field.
"It would be an absolutely unbelievable result for us to win at Stamford Bridge and do the double over Chelsea in the same season. It's one of those things that happens rarely, for you to be able to beat a top side home and away.
"It's a great incentive for the lads to go out there and achieve that on Sunday."