Mark Noble handed them a third-minute lead from the penalty spot and Joey O'Brien doubled the advantage with an opportunist strike 23 minutes later.
Although Norwich scored a late goal to set up a nervy finish, the hosts were worthy winners.
Allardyce said: "We have experienced the joy of winning again - it has been a while.
"It has been a torturous few games for me, but we have got what we deserved today, which is a win.
"It's a shame that the score didn't reflect the performance. The win was very important to us, but it should have been about 4-1 or 5-1 based on the amount of chances created and missed.
"Apart from the goal they scored in the last minute, we have got the vital win were looking for and we deserved, right from the start to the very end."
West Ham were severely under-strength for the game, with Kevin Nolan, James Collins, Mohamed Diame and Andy Carroll all missing.
But Allardyce felt their presence was not missed courtesy of the big impression made by his replacements.
He added: "We had a whole host of players missing, but what a performance when we were down to the bare bones. What a performance from the players that played today. Fantastic.
"It was an outstanding performance under the circumstances and an outstanding result. We did it with really good football and some good finishing."
Norwich were enraged by Mark Clattenburg's decision to award a penalty for Ryan Bennett's slight pull on Winston Reid, but Allardyce felt the referee was correct.
"Sometimes the referees give it and sometimes they don't," he said.
"He was right in front of the referee and the referee has got a very good view of it.
"The referee had a clear view of it. He saw this one and he gave it, and I thought it was the correct decision."
West Ham now sit 11th in the Premier League, but Allardyce insisted he was still keen to improve his squad during the January transfer window.
"We will try," he said. "We have been trying very hard over the last few weeks to line up [some players] in the short term.
"We will wait and see as January develops."