Match official Anthony Taylor found himself firmly in the spotlight with two contentious decisions in a dramatic clash at Upton Park.
Taylor over-ruled his assistant in first-half added time to allow Guy Demel's equaliser to stand even though Simon Mignolet had spilled the ball after being caught by Andy Carroll's flailing arm.
And the man in the middle then awarded Liverpool their second penalty of the game, with captain Steven Gerrard converting both, after Jon Flanagan went down under Adrian's challenge, although the West Ham keeper appeared to get a touch on the ball.
Allardyce feels changes need to be made to prevent the kind of errors which he feels have marred the top-flight this term.
He told Sky Sports: "Unfortunately we're talking about the officials rather than talking about the game, sadly, at the end of it.
"We're faced with this difficult situation where we're talking about it nearly every weekend and that's my real concern.
"Having had professional referees for such a long time, we clearly don't have enough and we don't have enough quality.
"We need to do something about it at the very, very top and do something as quickly as we can.
"They try their best, they don't go out there to make mistakes. They go out there to be as honest as they can be. We need officials to make fewer mistakes than they're making this season."
The Hammers boss added: "I feel really let down, the lads feel really let down. No-one gave us a chance today and we took Liverpool right to the wire.
"There were more than two major decisions (and), for me, (it is) about the overall control of the game from the officials.
"There is a linesman that gives one that is far, way and beyond his reach, then it gets turned over by the referee and looks like a foul on Andy.
"Then they get a penalty that's not a penalty. Flanagan is going down before Adrian has even played the ball. Adrian plays the ball, the referee is not in a great position. He can't be 100 per cent certain, so guesses.
"Both teams want to play the game and have the result determined on how we've played, not by what the referee's decisions were and the big mistakes he made today."