David Mooney, Elliott Omozusi, Gary Sawyer and Lloyd James netted their spot-kicks for the home side and although Leon Clarke and Jordan Clarke were on target in reply for the Sky Blues, Franck Moussa had his effort saved by Jamie Jones while John Fleck missed the target.
It ensured the O's - who top League One after 10 matches - triumphed 4-2 after their second-round tie at Brisbane Road finished goalless following 90 minutes.
"I'm very disappointed for the players because they gave so much to the game," Pressley said.
"I felt we should have won and that's the really disappointing aspect. I thought we performed very well for the 90 minutes, it was a very tight game and there weren't a lot of opportunities for either side. But I thought that we were the dominant team in terms of possession and always looked very much in control of the game.
"On the encouraging front we've kept another clean sheet away from home and that's the second in successive games. Our players are really growing all the time. It's not easy for a young team to come to a place like this and handle themselves against the league leaders.
"It would have been nice to have won the game but the margins are so fine that it's very unfortunate that a game like this has to be decided on penalties.
"On the whole, I thought that our work-rate, our desire and our willingness to help each other was terrific and I'm really, really proud of how the team is progressing and we are moving in the right direction."
And Orient assistant manager Kevin Nugent admitted Coventry had provided the London club with a stern test.
"Make no mistake, Coventry are a good side," Nugent said.
"They moved the ball around well and I thought our discipline in contending with that was fantastic. We had to keep a really good shape and to be honest, we created the best chances in the game.
"Coventry made it difficult for us because they had a lot of possession."