The Northern Ireland international midfielder, 24, was not offered a new deal by Scunthorpe earlier this month and he has opted to join the rebuilding operation being completed by Millers boss Steve Evans.
O'Connor, who will finalise his move on July 1, told the club's official website: "I spoke to Steve Evans, met up with him on Wednesday, and within 10 minutes I'd made my mind up.
"It's stepping down a division coming to League Two but it wasn't a tough decision because everyone I spoke to about the club said it was going places."
O'Connor joins Kayode Odejayi, Daniel Nardiello, Ian Sharps, Nicky Hunt and Scott Shearer in joining the Millers.
