Garry Thompson's first-half strike took the League Two club to within three minutes of winning the game in normal time before Thomas Vermaelen's late header.
But after no further goals in extra time, Vermaelen then missed the decisive penalty to ensure Bradford won a record-breaking ninth successive shoot-out and reached the semi-finals for the first time in their history.
For Arsenal, it was a humiliation of epic proportions in an already disappointing season after Arsene Wenger had named an almost full strength team when searching to end the club's infamous seven-year trophy drought.
The 23,971 sell-out crowd, Bradford's biggest since 1960, had groaned as the teams were read out but they were soon cheering when their top-scorer, Nahki Wells, out-muscled Vermaelen and forced a sprawling save from Wojciech Szczesny.
Hope turned to jubilation for the Bradford fans in the 16th minute when a free-kick led to Thompson's goal as he poked in at the back post.
As the temperature on an icy night dropped, so did Arsenal's spirits, with Per Mertesacker being regularly beaten in the air by James Hanson.
The Gunners' frustrations continued in the second half, with Jack Wilshere flashing a shot well wide, then Gervinho mis-controlling when through on goal.
But the desperation finally paid dividends with three minutes of normal time remaining, when Santi Cazorla whipped in a delicious cross for Vermaelen to head home from close range.
Extra time was required and Cazorla nearly won it with a swerving effort from fully 30 yards that struck the crossbar but Bradford held out for the shoot-out.
Arsenal were always on the back foot from when Cazorla had their first spot-kick saved by Matt Duke, even more so when Marouane Chamakh hit the post.
Bradford could even afford to see Szczesny deny Ritchie Jones and Stephen Darby before Vermaelen hitting the post sent Valley Parade into ecstasy.