The Robins were up against it for much of the second half as Wednesday dominated, but they snatched all three points after a fine solo goal from Nicky Maynard with 10 minutes remaining.
Defeat will be difficult to take for Alan Irvine's men as they wasted a precession of second-half chances, with City keeper Stephen Henderson in inspired form.
The 21-year-old made two fine saves to deny Luke Varney, while Leon Clarke, Marcus Tudgay and Darren Purse were all guilty of missing the target.
The defeat, Wednesday's first in seven games, means they slip into the relegation zone with results going against them elsewhere.
Life could not be better for Bristol City on the other hand as they extend their unbeaten run to five games, with caretaker manager Keith Millen only improving his chances of landing the position on a permanent basis.
It was a bright start and both sides had opportunities to go ahead in a chance-laden first 15 minutes.
Tom Soares and Tudgay both fired over for Sheffield Wednesday - Tudgay's was a particularly guilty miss as he was unmarked at the far post - while home goalkeeper Lee Grant produced a miraculous save to deny Lewin Nyatanga's prodded effort.
Wednesday had Grant to thank again shortly after as he denied Maynard with another fine stop.
Danny Haynes had pace to burn down the right, beating Mark Beevers before pulling back for his strike partner, whose low effort was parried by Grant when it should have been converted.
The chances kept flowing and it was Wednesday's turn to create an opening when Clarke closed in on goal, but from an acute angle he chose to shoot rather than square for Tudgay and his effort sailed over the crossbar.
The hosts made an attacking start to the second period and Henderson twice kept his side on level terms.
First he got down well to save Varney's low effort after his surging run into the box and then he had to acrobatically tip Darren Potter's curling free-kick over the crossbar.
And after a concerted spell of home pressure, Henderson produced another fine save as he rushed out to deny the dangerous Varney as Wednesday stepped up their search for an opening goal.
Bristol City could not find any momentum and Wednesday continued their dominance and wasted further chances to go ahead when Purse headed over and Clarke hooked wide.
The Owls onslaught continued and after Soares had curled just wide, they were denied a clear penalty when Purse was felled in the box.
But then the hosts were hit with the sucker-punch in the 81st minute when Maynard collected the ball outside the area and went on a weaving run, which saw him beat several defenders, before coolly slotting home for his 17th of the season.
Wednesday were stunned and it could have been even worse but David Clarkson's low shot hit the post and went to safety in the closing stages.