James Fowler had put the hosts in front on 14 minutes but his effort was cancelled out by Billy McKay's six-minute brace to give the Highland visitors a 2-1 interval lead.
Inverness midfielder Jonny Hayes was sent off for a second yellow card just before the hour mark and Kilmarnock quickly capitalised, equalising through Michael Nelson.
Shiels netted four minutes apart in the closing stages to seal the points, although the hosts had to endure a nervy end to the match after Steve Williams reduced the deficit right on 90 minutes.
Kilmarnock manager Kenny Shiels named an unchanged side, sticking by the players who won at Hibernian, while forward Dieter van Tornhout returned from injury to take his place on the bench.
Visiting manager Terry Butcher was able to make four changes, welcoming back goalkeeper Ryan Esson from injury and Graeme Shinnie, Ross Tokely and Greg Tansey from suspension.
Inverness won the game's first corner on three minutes from which Tokely headed past the right-hand post when it was delivered from the left by Aaron Doran.
Hayes was also off target moments later when he tried his luck from the edge of the box.
Kilmarnock took the lead with their first attack of note on 14 minutes. Good work by Shiels saw him play in Danny Racchi, and when his shot from inside the box was parried by Esson, Fowler was on hand to lap up the rebound from close range.
Kilmarnock came close to doubling their lead after a free-kick was awarded for a foul on Gary Harkins.
The set-piece was curled out to the right by Racchi and Lewis Toshney saw his diving header go just wide of the target.
Inverness equalised just before the half-hour, Hayes' cross from the right finding its way to Tansey whose volley hit the bar, but when Kilmarnock failed to clear their lines the ball found its way to McKay, who netted from six yards.
McKay added a second just six minutes later.
Kilmarnock had been piling on the pressure but the Highlanders launched a counter attack, Hayes leading the charge down the left and crossing for McKay whose delicate header nicked the inside of the left-hand post before creeping over the line.
Both sides tried their luck early in the second half, Shiels racing up the left for Kilmarnock and trying a shot which went wide, while Tansey forced home goalkeeper Cammy Bell to smother his shot.
Doran was only denied by a timely tackle from Liam Kelly on 54 minutes as he sought to apply the finishing touch to a promising ball from Graeme Shinnie, and moments later Hayes crashed an effort off the bar for the visitors.
That was the midfielder's last contribution before he was sent off on 57 minutes for his second booking of the game, his foul on Ben Gordon adding to a yellow card five minutes before the interval.
Kilmarnock had already been pressing for an equaliser and brought on Van Tornhout and William Gros as they continued their search against the 10 men.
They were rewarded within nine minutes when Nelson made it 2-2 with a precise back-post header from Racchi's cross from the left.
Kilmarnock applied continuous pressure in the final quarter and Kelly tried a speculative 30-yard effort which came close to sneaking in.
Shiels restored his side's lead 10 minutes from time, weaving down the left and into a busy penalty box before unleashing a low shot which went in at the far post.
The Kilmarnock forward then sealed the win with a second goal four minutes later after Gros squared the ball into middle of box where Shiels arrived at pace to sweep home decisively.
Inverness pulled a goal back in the final minute when the home defence allowed Williams to fire home.