The first 10 spot-kicks were all scored but Accies goalkeeper Michael McGovern saved the 11th penalty from Well substitute Simon Ramsden and Hamilton’s teenage sub Craig Watson stepped up to net the winner and secure an away tie against Aberdeen.
The teams could not be separated after 120 goalless minutes in the Lanarkshire derby, although both sides finished the game with 10 men after the hosts had Stephen Hendrie sent off in the second half with Motherwell’s Mark O’Brien following him down the tunnel in the first half of extra-time.
There was a lively start to the game and Louis Longridge had the first decent chance to break the deadlock, but his shot from the edge of the area was well blocked by Stephen McManus.
Tony Andreu’s weak header then failed to test Dan Twardzik in the visitors’ goal and Ali Crawford spurned another chance for the hosts when he was foiled by O’Brien’s clearance after being played through on goal.
Josh Law wasted an excellent opportunity for Motherwell at the other end, but Hamilton carved out two good openings with Crawford blazing over from long range before Twardzik made an acrobatic save to keep out Longridge’s 20-yard effort.
Jack Leitch’s half-volley was not too far wide of the target for the Steelmen who then missed out as an unmarked John Sutton headed over.
The end-to-end action continued before the interval as Andreu hooked a shot over the bar and O’Brien headed Iain Vigurs’ free-kick too high, but it remained goalless at half-time.
It was a much tighter contest at the start of the second half and the hour was almost up when Accies striker Eamonn Brophy had the first decent chance, but he could only poke the ball wide.
Hamilton were reduced to 10 men in the 66th minute when Hendrie saw red after committing a second bookable offence with a challenge on Lionel Ainsworth.
Motherwell had a good chance to take the lead six minutes later but Sutton slid the ball just wide from Vigurs’ corner.
Vigurs then must have thought he was certain to open the scoring with a powerful volley from close range but he was denied by a miraculous save from McGovern.
Twardzik made a more routine save at the other end from Crawford’s long-range effort before Well went close again when sub Henrik Ojamaa fired inches wide.
Sutton looped a header over with a minute of normal time left and Motherwell remained in command as extra-time got under way, with Leitch curling a shot just wide.
The hosts had an amazing let-off in the 95th minute when Law was played clean through following a one-two. The midfielder shot against the bar and Ainsworth volleyed the rebound against the post.
Hamilton sub Darren Lyon then fired just wide before the visitors were also reduced to 10 men when O’Brien received a second yellow card in the 102nd minute after tangling with Crawford.
Both sides then struggled to create chances as they tired in the closing stages, although the Accies nearly won it late on when Longridge was narrowly off target, but it did not matter in the end as they prevailed in the shootout.