The Hammers were beaten 3-0 at home to London rivals Chelsea, and were booed off at half-time and full-time by the home support.
The Hammers have two wins all season - on the opening day against Cardiff and, memorably, at Tottenham.
The defeat by Chelsea was their fourth at home this season - the same number as the whole of last season and Allardyce admits he understands the increasing frustration in the stands and puts recent performances down to 'fear' in the team.
"If they (the supporters) don't see the sort of determination and passion and desire they think they should get they show their disapproval," Allardyce said.
"I thought the players tried as hard as they can, but they're playing with a bit of fear at the minute. And that makes them look like they're not playing with the passion that is normally seen."
West Ham have two more derbies coming up, and Allardyce says they need to turn things around quickly.
"When people say there's a pressure game at the top - Manchester United versus Arsenal or Chelsea versus Manchester United or Manchester City - it's not as big a pressure as when you're in the bottom three or four," Allardyce added.
"It's much greater pressure because it's all about 'relegation this', 'relegation threatened that', all the money you lose.
"That becomes a constant flea in the players' ear. They've got to dismiss all that, keep a clear mind, keep a clear head and focus on playing how they know they can play.
"It's my job to pick the players up and remind them about previous performances we've had home and away this year, make sure we get to that level and stop those fears creeping in."