Niko Kovac, a former Croatia midfielder and coach of the junior national team, will be Stimac's temporary replacement.
Croatia qualified for the World Cup play-offs despite the defeat at Hampden Park and Stimac said after the match that he would offer his resignation.
The 46-year-old said he would step down because of "a moral responsibility" for Croatia's two consecutive defeats, 2-1 against Belgium at home last week and against Scotland.
Stimac took over from Slaven Bilic. The duo were Croatia defenders in the team that reached the 1998 World Cup semi-finals.
Croatian Football Association (HNS) president Davor Suker said: "The HNS executive board decided unanimously to relieve Stimac of his duties and terminate his contract while the body has also decided to name Kovac as his successor.
"It was a difficult decision because Stimac is a personal friend and a former Croatia team-mate but a string of poor results and performances made it inevitable.
"Kovac will take over as the caretaker coach for the upcoming playoff but our intention is to offer him a long-term contract whether or not we reach the World Cup finals in Brazil."
"I am aware that Stimac offered his resignation in a statement to the media after the game but we received nothing from him in written or verbal form on the flight back or afterwards, hence he was sacked.
"Kovac is full of enthusiasm and we are hoping that he will steer us to Brazil because he is the right man for the hard work that lies ahead in a very short time frame."
Croatia will go into next month's play-offs but the former Derby and West Ham defender had spoken to president Suker after the Scotland defeat to put his fate in his hands, after becoming fed-up with the criticism that has followed him during the whole campaign.
He said on Tuesday night: "It is a bad defeat. I offered my mandate (resignation) to the president of the Croatian football federation because of the latest results of our team, because everything that surrounds us.
"Never mind that we qualified for the play-offs, I want to show some morale, certain responsibility and if there is anyone in the federation that does not believe in my work I want to give them the chance to change.
"They will have a meeting to discuss it."
Stimac continued: "It is all about the positiveness surrounding the football in our country.
"It seems we are not happy in Croatia to qualify for the play-offs.
"We were not happy after the first game even though we beat Macedonia, we were not happy in the second when we drew against Belgium or beating Wales 2-0, they said it was supposed to be 15-0.
"After Wales we went to Macedonia and beat them 2-1 and they said that was not good because they were on the up, we beat Serbia 2-0 and they said we barely survived and the goalkeeper saved us.
"We beat Wales 2-1 and they said we were lucky. That's how it works in Croatia.
"There were a lot of people against me from the beginning."