McSheffrey headed home Aron Gunnarsson's right-wing cross in the 86th minute to chalk up Coventry's fourth league win of the season.
The Sky Blues' recent signing Marlon King had earlier been forced off in his first league start since serving a jail sentence.
Doncaster striker Billy Sharp struck his third goal of the season to cancel out Gunnarsson's second-minute opener for Coventry, who created fewer goalscoring chances than their visitors, but McSheffrey's strike proved decisive.
King replaced Lukas Jutkiewicz in Coventry's starting line-up, while Daniel Ward, on loan from Bolton, stepped in for Carl Baker. Defenders Adam Lockwood and Sam Hird were recalled by Doncaster.
Both players had barely settled when Gunnarsson fired his second goal in as many matches for the Sky Blues in the home side's first attack.
The Iceland international latched on to Clive Platt's ball across the face of goal and converted from six yards.
Doncaster responded in the 13th minute with Martin Woods dragging a shot wide from the edge of the penalty area and then volleying past the post from 20 yards, while James Coppinger's close-range effort was comfortably saved by Keiren Westwood.
Coventry almost doubled their lead in the 22nd minute through Ward's thumping drive, but Doncaster goalkeeper Neil Sullivan pushed the ball away at full stretch.
Rovers were then dealt a blow when Shelton Martis was forced out of the action and replaced by Mustapha Dumbuya, while King spooned his first effort weakly over the crossbar.
The visitors drew level two minutes before half-time when John Oster's superb raking pass picked out Billy Sharp and the Rovers striker drilled home the equaliser, low from 12 yards.
King was denied a goal when Sullivan was alert to tip over the striker's cross-shot, while Westwood pulled off an even better save at the other end to keep out Hird's header from Oster's corner.
Sullivan then blocked Platt's goalbound header before King's night was cut short through injury in the 65th minute, with Jutkiewicz taking his place.
Doncaster midfielder Brian Stock, a 70th-minute substitute for Dean Shiels, soon made an impact as his volley was blocked by Lee Carsley, while Oster's 12-yard shot struck Ben Turner and Westwood held Hird's stabbed effort.
The game appeared to be heading for a draw, but in the 86th minute Gunnarsson sent over a pinpoint cross from the right and McSheffrey, on since the 61st minute in place of Ward, powered home his header into the bottom corner from six yards.