The League One side were just a minute from earning a fourth-round tie with Barnsley before Dean Cox's fortuitous equaliser forced the extra half-hour.
And with a penalty shoot-out looming, Tom Cairney struck the winner in the 117th minute to snatch victory.
"All credit to Orient, they have given us two really difficult games and if we're honest we could be out of the cup. But we're not," said Bruce.
"We had some good chances to put the game to bed, but when it's just 1-0 you are always on edge and they've scored with the last kick, really, to take it to extra time. And then it's anybody's game.
"You hope a little bit of quality comes through in the end but fair play to Orient, they took us to two really hard cup ties and on another day could have gone through."
But Bruce also revealed that midfielder Sone Aluko faces a lengthy spell on the sidelines with an Achilles injury.
"Sone Aluko looks to have a slight tear to his Achilles, and could be out for around six weeks," the Tigers boss added.