The Hammers surrendered their first-half lead given to them through Ravel Morrison to come away with nothing from the game.
The hosts fought back with goals from Gary Hooper, Robert Snodgrass and Leroy Fer to seal victory for the Canaries.
"What disappoints me more is that the confidence we have gained from our away-performance results was there for all to see in the first half," Allardyce told Sky Sports 1.
"I expected that one of the things we'd been short of was getting the first goal and going ahead, and I thought once we got that the lads were going to be as good as they've been all season - having conceded one goal away from home from a penalty that shouldn't have actually been a penalty.
"But then our own errors and our own mistakes have let Norwich back into a game that they shouldn't have got back into and we've ended up with no points instead of picking up points and a win.
"We've only ourselves to blame.
"When we needed that all-important clean sheet today to get the three points it's deserted us on our own failings rather than Norwich's good play."