Attacking midfielder Toivonen, who has 40 caps for Sweden to his name, scored 14 goals in 46 appearances for Rennes.
The 29-year-old began his career in his homeland with Degerfos IF before playing for Gothenburg and then at Malmo.
He moved to PSV Eindhoven – where he played under current Sunderland boss Dick Advocaat – winning the KNVB Cup and Dutch Super Cup, before joining Rennes in January 2014.
Advocaat told the club’s website: “I worked with Ola at PSV so I know him well.
“He is a national team player with a lot of experience, an attacking midfielder who can score goals and is good in the air – he will bring something different to the team.”