Smith headed in his ninth goal of the season after stretching superbly to meet fellow substitute Kirk Millar's cross in the 79th minute.
Two nervous sides played out a disappointing goalless first half at Gigg Lane.
Oldham's Cristian Montano saw an early shot saved well by Bury goalkeeper Cameron Belford, while the hosts responded with striker Andy Bishop heading inches wide from Tom Soares' cross.
Lee Croft fired a low shot wide for the visitors, while skipper Robbie Simpson headed narrowly over the top from Montano's cross.
After the break Zac Thompson ought to have doen better for Bury when he drove a shot straight at Oldham goalkeeper Dean Bouzanis.
Bury's on-loan striker Jonson Clarke-Harris then leapt superbly, but saw his header trickle just past the post.
Lee Barnard fired a decent effort over the top for the visitors, but Smith was their late hero as Oldham, two points clear of the relegation zone, edged a little closer to safety.