West Ham were teetering on the brink of the relegation zone going into the game, level on points with their opponents, but victory sees them rocket up to 10th in the top flight standings.
Allardyce was made to sweat for the three points as West Ham left it until the final six minutes to break the deadlock.
Substitute Mohamed Diame provided a precision cross for James Collins to head home his first Premier League goal in six years.
With Norwich committing all their men to attack, Diame doubled the tally in added time, racing the length of the field before slotting beyond John Ruddy, with the aid of a deflection.
"We are delighted in the end because it was a hell of a game Norwich gave us tonight," Allardyce told Sky Sports.
"It was a really hard fought game but I think in the end our goalkeeper has kept us in the game and then the set pieces have put us on the right track.
"Obviously when you score that late on it's difficult for the opposition to come back and when they throw everybody forward, Mo Diame was able to finish it off for us."
West Ham are now unbeaten in four and Allardyce admits the mood in the dressing room is one of elation.
"It was a great end to a great run of fixtures for us," he added.
"We have a few days now where we can regroup, there is a lot of tired legs but not so many tired minds.
"We are in a great mood at the moment and this is a really big and hard-earned three points for us."
Another clean sheet for Adrian in the West Ham goal further underlined their defensive authority, which has seen them keep sides goalless in half of their fixtures this term.
Goals have been hard to come by for the Hammers, but their recent upturn in form has impressed Allardyce as both defence and attack click into place.
"Thirteen is an unbelievable figure when you've been struggling in the league," he said.
"It's all been about not enough goals but now we've started to score and for us it's about getting both ends right now.
"We've always had it defensively right apart from when we had a group of injuries to the central defenders and full-backs but now the defenders are back."
He added: "We are defending extremely well and we are starting to take our chances.
"We didn't have many tonight so it's nice to see us taking them when we don't have many."