Despite falling behind after four minutes, Kevin Blackwell's side gained command through strikes from Danny Mayor, Andy Procter, Nathan Cameron.
Substitute Jessy Reindorf then completed an impressive performance by scoring late on.
Bury had made a bumbling start when Procter turned Terry Gornell's weak shot into his own net.
But they hit back in fine style after nine minutes as Mayor cut in from the flank to unleash a stunning 30-yard shot that found the top corner of the net.
Momentum in their favour, the Shakers went ahead in the 12th minute when Procter made amends for his earlier error by heading home a Craig Jones corner.
Jones also supplied the ammunition for Bury's third goal in the 17th minute, crossing to the near post where Cameron poked home.
Cheltenham struggled to recover from that whirlwind spell, although Russell Penn hit the post with a shot just after half-time.
Sub Reindorf put the cherry on the cake by heading home Mayor's cross in the 76th minute.