Albion squandered the two-goal lead given to them by Billy Kee and Aaron Webster in the first half as their League Two rivals hit back with late goals to send the third-round tie into extra time, where a long-range effort from Stephen Darby after 115 minutes won it for the Bantams.
"Obviously, we are very disappointed to go out of the cup," said Rowett.
"Bradford's first goal was a poor goal to concede and when you concede after 83 minutes there is a fear that they might nick another one.
"I can't fault the players - I thought they were terrific.
"The team worked their socks off and gave a terrific performance against a very good side, although I was disappointed with one or two decisions and I felt we didn't get the rub of the green."