Rovers led an enthralling tie 2-1 and 3-2 but Tom Cairney was then sent off and David Amoo scored his second goal of the night with six minutes of extra-time remaining to force penalties.
Blackburn even had the edge in the shoot-out but Chris Taylor and Josh Morris then missed from the spot, allowing teenager David Symington to win it for the hosts.
"I'm disappointed because we came here to try and get through to the next round," said Bowyer. "There were two poor pieces of defending.
"We should have been professional and seen it out, and we've got to learn quicker than that. However, full credit to Carlisle.
"We are extremely disappointed but we have just got to dust ourselves down and focus on the next game."
Abbott: Kids showed no fear
Jubilant Carlisle manager Greg Abbott said: "We had a lot of kids out there and they were fantastic tonight. They showed no fear and should be proud of themselves.
"It was a terrific game after Saturday, when we were poor (in a 5-1 loss at home to Leyton Orient), but that's what the Cup is all about. There were so many positives and I think we deserved our little bit of luck in the game.
"These lads are going to get better and this game was important for the club. We won a little bit of respect back."