United looked out of things after Cillian Sheridan fired Killie into a 2-0 first-half lead and Paul Heffernan (53) had made it 3-1 after Gary Mackay-Steven (48) reduced the deficit.
But captain Daly struck in the 56th minute before Sheridan crucially had what would have been his hat-trick goal ruled out for offside in the 77th minute.
Daly made the most of that reprieve when he stooped to head home from Barry Douglas' perfect cross to send the home fans away happy and with their money's worth from a thrilling encounter.
The visitors began the brightest and came close early on when Borja Perez tried his luck from 30 yards with a shot which Radoslaw Cierzniak was relieved to see clear his crossbar.
Killie took the lead after eight minutes when Sean Dillon fouled Sheridan 30 yards from goal and the striker curled an inch-perfect free-kick beyond Cierzniak and into the top corner.
Sheridan doubled his tally on the half-hour mark when Gary Harkins produced a deftly weighted pass which split the United defence, allowing Sheridan to run in behind and dink a left-footed finish beyond Cierzniak and into the opposite corner of the net.
Within minutes of the restart Mackay-Steven sparked a comeback with an unstoppable strike from 30 yards, but Heffernan struck minutes later when Cierzniak failed to deal with a ball into the box and Heffernan looped home a 16-yard header.
Daly then headed past Killie goalkeeper Cameron Bell from Douglas' 56th-minute cross to make it 3-2, before Sheridan was denied a hat-trick when he was ruled offside.
And United made the most of that decision in injury time when Daly produced a typically brave diving header from another Douglas cross to round off a courageous fightback.