Seferovic's winner capped a remarkable comeback by the top seeds in the group after fellow substitute Admer Mehmidi had levelled the game in the 48th minute just 121 seconds after being introduced as a half-time substitute.
Enner Valencia had given Ecuador the lead midway through the first half as he scored for the fifth successive game for his country.
Victory was Switzerland's first over South American opposition in the World Cup finals after four attempts, and the win had even Hitzfeld jumping up and down for joy in what is his farewell tournament.
Hitzfeld, who guided Borussia Dortmund and Bayern Munich to Champions League glory, said the win would do wonders for the morale of his team and set them up nicely for further games against France and Honduras.
"It was a dream end to the game for us and is very important for the morale of the team," said the 68-year-old German, who will be mindful that in 2010 the Swiss opened with a win over eventual champions Spain but ended up going out in the first round.
"It will give us some extra strength for the next two matches.
Seferovic's dramatic winner came after a brilliant block tackle by Valon Behrami in his own area instigated a break which was allowed to continue by an excellent decision by Uzbekistan referee Ravshan Irmatov after the midfielder was body-checked.
The move continued with Ricardo Rodriguez crossing for Seferovic to finish in the 93rd minute, sending the Swiss fans into raptures and deflating the South Americans, who had played their part in another World Cup attack fest, albeit lacking the quality seen in earlier encounters.
"I really thought this move was delightful," added Hitzfeld.
"We reacted intuitively and knew how to take advantage of the Ecuadorean mistake. It is marvellous to win in the last minute as it releases positive energy and is very good for the morale of the team."
Man-of-the-match Xherdan Shaqiri added: "We could have gone 2-1 behind but we managed to make a very good save and then scored up the other end.
"After half time we played a lot of simpler passes and we let the ball do the work and that created chances."