Tonge, 28, was released by Millers manager Steve Evans at the end of last season after six years with the South Yorkshire club.
Gulls boss Alan Knill has already said he expects players to arrive at Plainmoor in the coming weeks.
"I spoke to him and he came down to the area last week," Knill told Torquay's official website.
"He is somebody we like and somebody who knows the league very well.
"He's played for Barnsley and Rotherham, and we've put an offer to him. We are in negotiations with Dale and we will see where that goes.
"He plays as a right-back and is the sort of player we are looking for. I wanted somebody who knows the league and I could have easily brought in a younger player, but we wanted more experience at the back. He is also at an age where he still thinks he can be better."
Tonge played 185 times for Rotherham after joining from Barnsley in 2007. However, he managed just 11 appearances as Rotherham were promoted last season due to persistent injury problems.