Second-half goals from Franck Moussa and Clarke saw Coventry come from behind to win after Jim O'Brien had given the hosts a first-half lead.
Following a lacklustre display before the break, Coventry applied strong pressure after the restart and deserved to go through to the fourth round.
During a quiet opening spell, the ball fell to Barnsley's Jacob Mellis inside the area but striker Clarke got back to make an important clearance, then O'Brien shot well wide of the target from long range.
The first effort on goal came from Clarke, who fired in a far post volley which was saved by Luke Steele after getting on the end of an attempted headed clearance.
O'Brien opened the scoring in the 19th minute, lobbing goalkeeper Joe Murphy after receiving a long ball forward from Jean-Yves M'voto.
Murphy was called into action shortly after the goal, diving to his left to keep out Tomasz Cywka's 30-yard free-kick.
Cywka again came close to scoring when he curled a shot beyond Murphy, only to see the ball come back off the keeper's left-hand post.
Denzel Slager and Cyrus Christie both put long-range efforts straight at Steele early in the second half.
There was a flurry of bookings in a four-minute spell around the half-hour mark with Barnsley's Stephen Dawson and Coventry pair Clarke and Moussa all issued with yellow cards by referee Carl Boyeson.
The impressive Cywka put a couple of efforts wide and David Perkins tested Murphy with a shot from around 20 yards out.
Clarke went close to equalising midway through the second half when his shot clipped the bar after Steele failed to gather the ball from a corner.
Dawson had a chance to extend Barnsley's lead when Mellis slipped the ball into his path inside the area - but he sliced his shot wide from a good position.
As Coventry looked to put the home defence under pressure in search of an equaliser, Blair Adams shot wide from just outside the area and John Fleck had a low drive saved.
Moussa levelled with a fine finish 12 minutes from time, after running through the middle and outpacing a couple of chasing defenders.
Coventry skipper Carl Baker saw his free-kick deflected over, Clarke headed wide and Christie had a shot saved as the visitors continued to press.
It was no surprise when Clarke burst through the middle before firing confidently past Steele to complete a stirring comeback from the Sky Blues in the 89th minute.
The success was achieved without 15-goal striker Callum Wilson, who is out of action after suffering a dislocated shoulder in the New Year's Day League One victory at Rotherham.