Scott Rendell's 80th minute goal proved decisive in a clash between two sides separated by 85 places in the English football ladder.
The Blue Square Premier side's victory at Carrow Road means they are only the seventh non-league team to reach the fifth round of the competition since World War Two.
Norwich were not the only top flight team embarrassed today, as MK Dons won 4-2 at QPR.
Harry Redknapp, who made sweeping changes ahead of Tuesday's visit of Manchester City in the league, saw his team thrashed by their League One opponents.
QPR knocked MK Dons out in the third round of this competition last year, but Armand Traore's fourth-minute own goal and Ryan Lowe's 40th-minute effort gave the visitors control at half-time.
Ryan Harley (50) and Darren Potter (56) made it 4-0 before Jay Bothroyd and Fabio pulled two goals back inside the last seven minutes for the Premier League side.
Manchester City booked their place in Sunday's fifth round draw with victory in the early kick-off at Stoke.
Pablo Zabaleta struck the only goal five minutes from time as City repeated the scoreline which won the 2011 final against the same opponents.
Yet victory may still have come at a cost for Roberto Mancini's men after captain Vincent Kompany limped off in the first half.
Olivier Giroud, who netted twice in the midweek defeat of West Ham in the Barclays Premier League, repeated the feat as Arsenal won 3-2 at Brighton.
The Frenchman gave the Gunners a 16th-minute lead at Brighton, but Ashley Barnes' header levelled the scores 17 minutes later.
Giroud restored the visitors' advantage 11 minutes into the second half, only for Argentinian striker Jose Ulloa to level on his debut six minutes later.
However, Theo Walcott struck to win it for Arsene Wenger's side five minutes from time.
Johnny Heitinga's 90th-minute goal gave Everton a 2-1 success at Bolton. Steven Pienaar's scrappy 18th-minute goal gave David Moyes' men the lead before Marvin Sordell - whose two goals saw off Sunderland in their third round replay - squared things nine minutes later.
Reading eased into the last 16 with a 4-0 win over Sheffield United.
Noel Hunt opened the scoring in the sixth minute, Mikele Leigertwood's long-range goal against one of his former teams doubled the advantage five minutes before the break, before Hunt and Garath McCleary made sure of the win within nine minutes of the second half starting.
Blue Square Premier Macclesfield were unable to emulate Luton's exploits, Jordi Gomez's seventh-minute penalty giving Premier League Wigan a 1-0 win at Moss Rose.
There was late heartbreak for League Two Aldershot at Championship Middlesbrough.
Danny Hylton's 89th-minute goal appeared to have earned the Hampshire side a replay but Lukas Jutkiewicz added to his 83rd-minute effort with a last-minute winner.
Blackburn were convincing 3-0 winners at Derby thanks to goals from Colin Kazim-Richards (45), Scott Dann (66) and Jordan Rhodes (71), while Chris Dagnall's goal 20 minutes from time decided another all-Championship clash in Barnsley's favour at Hull.
Lee Novak's 74th-minute penalty gave Huddersfield, under the leadership of caretaker manager Mark Lillis following the sacking of Simon Grayson, the lead against Championship rivals Leicester.
However, Chris Wood earned a replay for the visitors with a goal eight minutes from time.