After a goalless first-half, caretaker boss Roberto Di Matteo saw Mata break the deadlock eight minutes after the restart in their fifth-round replay at St Andrew's.
Then a superb 60th minute strike from Raul Meireles doubled Chelsea's lead to secure a home quarter-final tie with Leicester. They could even afford to see a Mata penalty saved by City keeper Colin Doyle.
Di Matteo left Frank Lampard, Didier Drogba, Michael Essien and Daniel Sturridge out of his starting line-up although skipper John Terry was a surprise substitute after his recent knee operation.
There was little to choose between the sides in the opening period when Nikola Zigic posed problems for Chelsea.
But the visitors stepped up a gear after the interval and only poor finishing prevented them from adding to their tally as injury-ravaged City ran out of steam.
Chelsea were the first to threaten and Mata raced clear but opted to shoot with Salomon Kalou unmarked in the centre and Doyle turned his effort around the post.
A neat flick from Zigic found Wade Elliott in space behind the Chelsea defence but his low first time ball was in front of the run of Morgaro Gomis.
City were causing problems and Nathan Redmond got clear in the box after more good play by Zigic but David Luiz got across to intercept as he prepared to shoot.
Zigic was unhappy when caught by the boot of Luiz and he was left with blood pouring from his face and required lengthy touchline treatment.
Dangerman Zigic created another opportunity for Elliott and his shot on the turn was gathered by Petr Cech.
Zigic became the first player to be yellow carded after a late tackle on Gary Cahill.
Cech produced a superb save to keep his side on level terms after 39 minutes.
Zigic's glancing header from a Jordon Mutch cross had Cech back-pedalling and he finger-tipped his effort over the bar at full stretch.
Mata was Chelsea's most potent player but he shot straight at Doyle from the edge of the box.
Fernando Torres had his first opportunity on the stroke of half-time after a quick turn but he dragged his shot across the face of goal.
Chelsea enjoyed a spell of possession at the start of the second period and Pablo Ibanez produced a superb tackle to deny Mata inside the box after he played a one-two with Kalou.
But after 53 minutes Mata broke the deadlock for Chelsea.
Ramires made good ground down the right before sending in a cross which was challenged for by Kalou.
The ball broke invitingly for Mata who poked it past Doyle into the corner of the net.
Mata missed another great chance two minutes later when he headed wide at the far post from a deep centre by Branislav Ivanovic.
A superb strike from Meireles doubled Chelsea's lead after 59 minutes.
Ramires teed up the midfielder and his 25-yard strike flew past Doyle into the top corner of the net.
Torres was yellow carded for chopping down Jonathan Spector.
The Spaniard won his side a 67th minute penalty after he was brought down by Guirane N'Daw who was yellow carded.
But Doyle repeated his penalty save from Mata of the first tie at Stamford Bridge.
Daniel Sturridge should have made it 3-0 but mis-kicked with the goal at his mercy from Torres' centre.
Mutch had a great opportunity to reduce the arrears from Chris Burke's pass but lofted his shot over the bar.