The 51-year-old's future had been in doubt for the last 12 months but his fate was sealed following England's disastrous showing at the European U21 Championship in Israel this month.
England exited the competition without a single point, losing all three of their matches against Italy, Norway and Israel and scoring just once against their Norwegian counterparts.
Pearce indicated before the opening group game in Tel Aviv that he wished to extend his contract but the FA has decided to call time on his six years at the helm.
The former Manchester City manager took over the role in 2007 and guided the side to four consecutive European Championships.
FA chairman David Bernstein said: "I'd like to thank Stuart Pearce for his hard work. Nobody can doubt his commitment to the job throughout his time as U21 coach.
"He has done extremely well in qualifying for Finals tournaments, but after the disappointment of the past two tournaments, we believe it is time to change coach.
"On behalf of everyone at the FA I'd like to wish Stuart well for the future.
"It is important the organisation now considers the best approach for the role of U21 coach and how this works best within the overall England structure before making any new appointment."
Gareth Southgate is odds-on favourite to replace Pearce while former England coach Steve McClaren is also thought to be interested in the post.
Earlier on Tuesday, Football Association general secretary Alex Horne hailed Pearce's professionalism.
Horne told Sky Sports News: "Stuart has brought to his country and to the U21 side a huge amount of passionate leadership.
"Do not forget he has qualified for four tournaments in a row, which is unprecedented, and he qualified top of the group last time.
"Stuart has brought a huge amount of professionalism to the FA, not only with the U21s but also with his spell with the Olympic team."