The Black Cats lost 1-0 to leaders Manchester United in Saturday's lunchtime kick-off that left them without a win in eight games.
It was also a result that left them perilously one point above the drop zone with seven games remaining in the Premier League.
A statement on the club website read: "Sunderland AFC has announced that it has parted company with manager Martin O'Neill this evening.
"The club would like to place on record its thanks to Martin and wishes him well for the future. An announcement will be made in the coming days regarding a successor."
The former Wycombe, Leicester, Celtic and Aston Villa boss was appointed on 3 December 2011 in a three-year deal as replacement for Steve Bruce and guided the team to top-flight safety with seven victories in his first 10 games to finish in 13th spot.
Facing the might of Manchester United on Saturday was not the type of game to boost confidence for a team stuttering at the wrong end of the table with the contest settled by an own goal by defender Titus Bramble.
After the match at the Stadium of Light, O'Neill, whose record over 66 games at Sunderland was 21 wins and 25 defeats, had tried to paint a positive picture on their hopes to avoid the dreaded drop.
The 61-year-old Northern Irishman told Sky Sports: "Confidence is a major thing in the game and we are striving for it, but I thought the players can take a great deal of confidence from that second-half performance.
"We have to go and put it right, and we intend to do so. There is a determination in that dressing room to get the points. Nothing is too easy in this game, but we are up for the challenge."
Sky Sports understands that a new manager could be announced as early as Monday with former England boss Steve McClaren the current favourite to step in.
O'Neill had never made any secret of his boyhood affection for the club and the chance to manage them fulfilled a long-held dream.
His start was just what owner and chairman Ellis Short was looking for as a run of seven victories in his first 10 league games eased the club out of relegation trouble and sent spirits soaring.
The Black Cats ran out of steam towards the end of the campaign, but with their top-flight status secured and the promise of a summer rebuilding programme, attention turned to the new campaign with genuine optimism.
That only increased when Short provided O'Neill with a total of £22million with which to buy Scotland international striker Steven Fletcher and England winger Adam Johnson, and the American put his hand in his pocket once again in January to add Danny Graham and Alfred N'Diaye to the squad.
However, only Fletcher, who was this week ruled out for the remainder of the season with ankle ligament damage, has hit the ground running with his 11 goals a major contribution to the 31 points they have claimed to date.
Johnson in particular has failed to recreate the form he showed in his early days at Middlesbrough and then in spells after his move to Manchester City, while Graham is yet to find the back of the net in seven appearances.
Short has been unhappy with results for several weeks, but writing in his programme notes for Saturday's game, he called for a united front.
He said: "Of course, there is nobody that is happy with our current position and the way our season is going. The reality is that we have a fight on our hands with eight games remaining.
"We know that we are capable of playing better than we have been doing and we know that we need to begin doing so now.
"I can assure you that every person at the club is doing everything they can to ensure that the season ends on a positive note and when the season is over, we can take stock of what happened.
"Right now, however, there is only one thing that every one of us should be focused on, and that is doing our part whatever that is in helping this team to get the points we need.
"I'm very aware that our supporters, who have backed us week in, week out in such huge numbers, aren't happy and fully understand why.
"But right now, it is important for us all to be on the same side and get behind the team. Not being together will not help us to get results, so let's stand shoulder to shoulder and give the team our full support."