Substitute Mats Moller Daehli snatched a potentially priceless point for the visitors just moments after Thievy Bifouma looked certain to have grabbed a first home win for boss Pepe Mel at the other end deep in stoppage time.
City had fought back from 2-0 down to restore parity at The Hawthorns with Caulker grabbing the second, before a crazy finish to the game which left the Bluebirds' captain's head spinning.
"We have conceded a lot of late goals this season, but I'm just proud of the boys," defender Caulker told Sky Sports.
"The last little burst in the last 30 seconds we managed to get the equaliser which gives us a fighting chance to stay in the league.
"My head was still fuzzy (after Albion's stoppage time goal). All I wanted to do was rally the boys up - I knew there was still time to get that goal and fortunately we got it.
"It (the point) has given us a fighting opportunity although we wanted to come here and win today but under these circumstances, going 2-0 down early doors.
"I'm pleased with the spirit we showed but we have got to start getting better now if we want to stay in the Premier League.
"We've got six cup finals to go. I'm pleased, but next week against Palace is a massive one now."