O'Brien's first goal for the club gave City their first win of the season and condemned United to back-to-back league defeats.
The first real chance came for the visitors when Ben Davies' cross from the right was met by Michael Higdon in the six-yard-box and the striker's header forced an excellent point-blank save from home goalkeeper Ryan Allsop.
Allsop's opposite number Mark Howard was called into action soon after and his superb one-handed save kept out Danny Swanson's curling effort.
City began to grow into the game and Jordan Clarke missed an excellent chance when well placed at the back post.
The second-half was a tight affair, but the Sky Blues netted their deserved winner in the 76th minute.
Ryan Haynes' cross from the left was met by O'Brien, who was left unmarked to head in via a deflection off Blades defender Bob Harris from six yards.