On-loan Middlesbrough man Marvin Emnes was the match-winner with his first goal for the Swans, in his first game and with virtually his first touch.
The Dutch debutant was a half-time introduction at the Liberty Stadium and wasted no time showing his power and pace as he shrugged off the attentions of Curtis Davies to latch onto Darren Pratley's throughball and fire past Chris Weale.
In-form Swans winger Scott Sinclair made the points safe in the 94th minute with his 10th of the season from Emnes' cross, having seen an earlier effort rebound out off the underside of the crossbar.
All that came after a competitive first half which lacked any real quality.
Eriksson had taken four points from his opening two games in the Foxes dugout, taking over from former Swans boss Paulo Sousa who was sacked after less than three months in charge.
The Swede handed a debut to new loan signing Greg Cunningham at left-back, with Lloyd Dyer and Matty Fryatt also coming in.
But there was no place for Darius Vassell after his recent arrival as the striker looks to improve his match fitness.
Swans boss Brendan Rodgers made one switch from the starting XI who held QPR to a goalless draw in midweek, Joe Allen coming in for David Cotterill.
A crowd of 14,651 were treated to a lively opening with Swansea starting on the front foot.
Nathan Dyer failed to connect with Stephen Dobbie's teasing ball across the the six-yard line, before the latter teed up Pratley in space in the penalty area but he could only fire over the bar.
The game was 16 minutes old when the visitors posed their first threat, Swans keeper Dorus de Vries producing a good stop to tip Fryatt's header around a post.
Counterpart Weale scrambled to hold onto Pratley's drive at the second attempt before Allen fired narrowly wide with a rasping effort from 30 yards.
But it was Leicester who came closest before the interval when De Vries made another fine save to tip Davies' goalbound header over the bar just minutes before the whistle.
Rodgers turned at the break to Emnes , who came on in place of Dobbie.
And the Dutchman made an immediate impact as he showed his strength to see off Davies and run onto Pratley's neat throughball before beating Weale from the edge of the area.
Swans defender Alan Tate was next to go close with another long-range strike for the hosts on the hour mark before Wales forward Andy King headed off target for the Foxes.
Winger Sinclair came within inches of doubling the home side's lead as the encounter approached the closing stages when his effort bounced off the underside of the bar and away to safety.
There was still time for Sinclair to get his name on the scoresheet, though, as Swansea came away with the points.