The Sky Blues looked on course to win it normal time when Harry Maguire scored an own goal 12 minutes in the second half. But United pulled a late goal back through David McAllister taking the tie to penalties.
The first three spot-kicks were dispatched before Maguire saw his penalty brilliantly saved by Murphy. Carl Baker made it 3-1 on penalties to Coventry which put pressure on Ryan Flynn, who buckled and fired over.
That allowed John Fleck to send Robins' team through with a calmly-taken penalty.
Wilson, who has now watched his team lose the play-off final and go out of this season's Capital One Cup via spot-kicks, said: "I am bit peeved that we have gone out in the way that we did again. We seem to have a mental block when it comes to penalties.
"I didn't think there was too much between the two teams throughout the game. Was it always going to go to a penalty shootout? Possibly so.
"I thought the second half was better from our point of view and the bonus points we take from the game is some of the young lads who have come into the side against a strong Coventry team.
"They made it difficult for us and we did for them, so chances in the whole game were at a premium.
"It's a disappointment to go out because I thought we had a chance to progress and go a little bit further but really the priority is the league. That said, games like this are there to win and you try to win them."
