The diminutive midfielder stepped off the substitutes' bench 10 minutes from the end to head Dale's 84th-minute winner from Nicky Adams' cross.
Gray's winning goal came completely against the run of play - Dale goalkeeper Owain Fon Williams pulling off a string of fine saves as Bournemouth bombarded the visitors' goal.
Bournemouth, who will host Huddersfield in the play-off first leg next Saturday, took the lead with the fifth of the 14 corners they forced, with central defender Jason Pearce capping a commanding performance by heading in Rhoys Wiggins' 27th-minute corner.
Matt Done equalised for Rochdale with a 20-yard shot 10 minutes before half-time, but the visitors held on as Bournemouth piled on the pressure after the break.
Twice Fon Williams made last ditch saves to deny Anton Robinson and then Adam Smith as Bournemouth pressed forward, only to be mugged by Gray's headed winner six minutes from time.