Killie overcame off some early lethargy before Heffernan opened the scoring in the 34th minute, with Shiels doubling that lead two minutes from the break with a fine strike.
Dundee's goal came in the 63rd minute when Killie midfielder Liam Kelly failed to prevent Gavin Rae reaching Gary Irvine's cross and the ball sped past Cammy Bell to bring the game back to the boil.
However, the Ayrshire side held out to book a place against Hibernian at Easter Road in the next round although they will probably have to improve on this performance if they are to progress any further in this competition.
Former Dundee striker Gary Harkins made what was a surprise return to the Killie side given that he was reported to be out for six weeks with a broken ankle at the start of January.
He replaced Danny Racchi, who dropped to the bench, while left-back Rory McKeown came in for the ineligible Ben Gordon. The Taysiders were unchanged.
Killie boss Kenny Shiels had claimed Dundee were the favourites going into the game.
Those who believed it was a case of the Northern Irishman playing mind games would have thought again in the early stages as Dundee, spirited and determined, were more than a match for their SPL opponents.
After Garry Hay missed an early chance for Killie when he hooked the ball over the bar from 10 yards, Barry Smith's side sprang into action.
In the seventh minute striker Steven Milne shot straight at Rugby Park keeper Bell, before he volleyed wide from eight yards following a fine inswinging free-kick from Ryan Conroy, barely a minute later.
The first half arguably turned in the 21st minute when Harkins turned smartly and went past Craig McEwan and Ross Chisholm with some ease albeit his weak drive went straight to Dees keeper Robert Douglas.
Nevertheless, the first piece of real skill shown by a Kilmarnock player appeared to boost the home side and four minutes later Harkins got his head to James Fowler's diagonal cross at the back post but his effort went narrowly past the target.
Killie's opener was basic. Bell thundered a kick down the pitch, Harkins headed on and Heffernan ran onto the ball before drilling a left-footed shot low past the diving Douglas from inside the box for his 12th goal of the season.
The hosts' second goal owed everything to the splendid finishing of Shiels.
After a Dundee move broke down in their own half, Harkins set up the Northern Irishman who flicked the ball up and drove high past Douglas from the edge of the box.
The onus was on Dundee to respond and they came out fighting after the break, wide-man James McCluskey flashing the ball across the Killie six-yard box minutes into the second period.
Dundee chased and harried the Ayrshire side who looked uneasy again - trying, but mostly failing, to get their passing game going.
And just after the hour mark the visitors were back in the game. Rae went up for the header with Kelly from Irvine's right-wing cross, and the Dundee player was soon celebrating with the travelling fans behind the goal after the ball ended up behind Bell.
The play swung from end to end as the Dark Blues went all out for the equaliser but an increasingly nervy Killie side, in the end, held out without too many goalmouth scares.