The U's made light of being from a division below their opponents, deserving Berry's 86th-minute winner and will now host Sheffield United on Sunday.
Cambridge started the better side but Kwesi Appiah wasted a good opening when he blasted over from a narrow angle after he was put through by Liam Hughes.
Hughes then saw a free-kick get deflected wide before Bury's Craig Jones had a header comfortably cleared off the line by Ian Miller.
The non-leaguers broke the deadlock after 61 minutes as Berry swapped passes with Harrison Dunk before keeping his composure in finishing past Brian Jensen.
However, the hosts' joy was short-lived as Shaun Harrad thumped Andy Procter's cross into the top corner with a terrific header just over a minute later.
Their equaliser really woke the Shakers up and they came within inches of going ahead as Danny Hylton's header from Jones' centre came back off the bar.
But it was Cambridge who were celebrating late on as Jensen hesitated from Michael Tait's corner and Berry got in front of him to nod in.