The 61-year-old's departure was confirmed hours after Saturday afternoon's 1-0 home defeat by Manchester United, which left the club sitting just a point clear of the Barclays Premier League relegation zone with seven games left to play.
A club statement said: "Sunderland AFC has announced that it has parted company with manager Martin O'Neill this evening.
"The club would like to place on record its thanks to Martin and wishes him well for the future.
"An announcement will be made in the coming days regarding a successor."
A run of eight league games without a win which has yielded only three points ultimately cost O'Neill his job after a little more than 15 months at the helm.
The Black Cats' last victory came at Wigan on January 19, and with trips to European champions Chelsea and rivals Newcastle to come ahead of bogey team Everton's visit to the Stadium of Light on April 20, things could get worse before they get better.
O'Neill, of course, was the man whose appointment Sunderland fans had craved for years, and his arrival at the Stadium of Light in December 2011 was greeted with delight.
The Northern Irishman had never made any secret of his boyhood affection for the club and the chance to manage them fulfilled a long-held dream.
His start was just what owner and chairman Ellis Short was looking for as a run of seven victories in his first 10 league games eased the club out of relegation trouble and sent spirits soaring.
The Black Cats ran out of steam towards the end of the campaign, but with their top-flight status secured and the promise of a summer rebuilding programme, attention turned to the new campaign with genuine optimism.
That only increased when Short provided O'Neill with a total of £22million with which to buy Scotland international striker Steven Fletcher and England winger Adam Johnson, and the American put his hand in his pocket once again in January to add Danny Graham and Alfred N'Diaye to the squad.
However, only Fletcher, who was this week ruled out for the remainder of the season with ankle ligament damage, has hit the ground running with his 11 goals a major contribution to the 31 points they have claimed to date.
Johnson in particular has failed to recreate the form he showed in his early days at Middlesbrough and then in spells after his move to Manchester City, while Graham is yet to find the back of the net in seven appearances.
Short has been unhappy with results for several weeks, but writing in his programme notes onn Saturday, he called for a united front.
He said: "Of course, there is nobody that is happy with our current position and the way our season is going. The reality is that we have a fight on our hands with eight games remaining.
"We know that we are capable of playing better than we have been doing and we know that we need to begin doing so now.
"I can assure you that every person at the club is doing everything they can to ensure that the season ends on a positive note and when the season is over, we can take stock of what happened.
"Right now, however, there is only one thing that every one of us should be focused on, and that is doing our part whatever that is in helping this team to get the points we need.
"I'm very aware that our supporters, who have backed us week in, week out in such huge numbers, aren't happy and fully understand why.
"But right now, it is important for us all to be on the same side and get behind the team. Not being together will not help us to get results, so let's stand shoulder to shoulder and give the team our full support."
That stock-taking exercise was dramatically brought forward.
O'Neill was seemingly not expecting the development as he conducted his post-match press conference.
He said: "I am as buoyant now as - I was nearly going to say as I have always been.
"What is it - the half-full or the half-empty glass? I'm even going to the three-quarters full.
There's a real determination in the dressing room. The players are obviously disappointed, but they can take a lot from the second half.
"Yes, there is a determination about the players in there that we can do it."
It is understood the search for a successor is already under way and that the club hope to make a swift appointment.
However, whoever gets the job faces the task of galvanising a squad for an intensely uncomfortable run-in with the stakes higher than ever as a result of the cash implications of the new broadcasting deal.
Sky Bet quickly installed former England boss Steve McClaren as the 11/8 favourite to succeed O'Neill, with Mark Hughes and Paolo Di Canio next at at 6/1.
Martin O'Neill factfile
1952: Born Kilrea in Northern Ireland, March 1.
1971: Makes international debut as a substitute against the USSR. Joins Nottingham Forest from Distillery in October.
1978: Member of Forest's league and League Cup double-winning side.
1979: Wins a second League Cup winner's medal as Forest retain trophy with 3-2 victory over Southampton.
1980: Wins European Cup as Forest beat German champions Hamburg 1-0 in Madrid.
1981: After 285 league appearances for Forest, with 48 goals, joins Norwich in February for the first of two spells as a player at Carrow Road. After 11 league appearances and one goal transfers to Manchester City.
1982: Returns to Norwich in January after playing just 13 league games for City. Makes 55 league appearances, scoring 11 goals.
1983: Returns to Nottingham in August, joining Notts County, making 64 league appearances with five goals.
1984: Wins last of 64 international caps in 2-1 win over Finland.
1990: Appointed manager of non-league Wycombe after spells with Grantham Town and Shepshed Charterhouse.
1991: Guides Wycombe to victory in the FA Trophy.
1993: Wycombe win GM Vauxhall Conference, securing promotion to the Football League. Clinches another FA Trophy success.
1994: Leads Wycombe to promotion from Division Three at the first attempt.
1995: Becomes manager of Norwich in June.
Resigns on December 17 after a series of clashes with chairman Robert Chase over money for team building. Appointed manager of Leicester on December 21.
1996: Leicester beat Crystal Palace 2-1 at Wembley in the First Division play-off final, winning promotion back to the Premiership at first attempt.
1997: Wins Coca-Cola Cup final, beating Middlesbrough in a replay at Hillsborough.
Guides Leicester to ninth-place finish in Premiership.
1998: Finishes 10th in Premiership with the Foxes but O'Neill frequently voices concern over lack of funds.
1999: Leads Leicester to Worthington Cup final, where they are beaten by Tottenham, and to 10th place in the Premiership.
2000: February - Leads Leicester to another Worthington Cup win as they defeat Tranmere 2-1 at Wembley.
June 1 - Becomes Celtic manager.
2001: March - Wins first trophy in Scotland with a 3-0 CIS Insurance Cup final victory over Kilmarnock.
April - Celtic win Scottish Premier League title.
May - Beat Hibernian 3-0 to win Scottish Cup final and complete a domestic treble.
2002: April - Celtic secure the Scottish Premier League title again with a 5-1 win over Livingston.
May - Celtic lose 3-2 in extra time to Porto - managed by Jose Mourinho - in UEFA Cup final in Seville.
Celtic lose SPL title to Rangers on thrilling final day.
2004: April 18 - Regains SPL title with 1-0 win at Kilmarnock with six games to spare.
May 22 - Earns second double in four years with 3-1 win over Dunfermline in Scottish Cup final.
July - Delays arrival in America for pre-season trip to care for sick wife, Geraldine.
2005: May 22 - Two late goals see Celtic lose 2-1 at Motherwell - and O'Neill's hopes of a fourth title in five years disappear as Rangers win 1-0 at Hibernian.
May 25 - Club announce O'Neill will step down after Scottish Cup final.May 28 - Ends career at Celtic with 1-0 win over Dundee United in Scottish Cup final at Hampden Park.
July 4 - Confirmed as manager of Aston Villa.
2007: May - Villa finish 11th in Premier League.
Following departure of Steve McClaren rules himself out of England job.
2008: May - Villa finish sixth in Premier League.
2009: May 2 - Villa finish sixth again, a run of nine games without a win costing them Champions League place.
August 27 - Fails to take Villa to the group stages of the Europa League following a 2-2 aggregate draw with Rapid Vienna. Rapid progress on away goals.
2010: February 28 - Reaches his first final with Villa - but they lose the Carling Cup 2-1 to Manchester United.
April 10 - Villa lose 3-0 to Chelsea in the FA Cup semi-final.
May - Villa finish sixth in the Premier League for a third successive year.
August 9 - Villa announce O'Neill has resigned as manager with immediate effect.
2011: November 18 - Rules himself out of the Northern Ireland manager's job, but agrees to help find the new man.
December 3 - Appointed manager of Sunderland, the team he supported as a boy, on a three-year contract.
Leads Sunderland away from trouble by guiding them to a 13th place finish in the Premier League.
2013: March 30 - Leaves Sunderland after a 1-0 home defeat against Manchester United leaves the team one point above the relegation zone.