Mourinho has been tipped for a comeback at Stamford Bridge after spending a successful three seasons there earlier in his career between 2004-07, leading them to two Premier League titles, as well as the FA Cup and two League Cup successes.
The Real manager is being touted as a potential successor to interim manager Rafael Benitez in the summer, with widespread speculation suggesting he will leave the Bernabeu.
Ivanovic, who joined Chelsea the year after Mourinho departed for Inter Milan, would welcome the opportunity to work with the Portuguese coach.
But the Serbian defender insists the players are ignoring the speculation for now.
"I think, for a top club, for a big club like Chelsea, we need the best from the players and from the manager and maybe one day it will be a pleasure to work with him," he told Sky Sports.
"We'll see. You never know and it's not time for us to speak.
"He was a success and for him personally he's said that Chelsea is his home and that he loved the club. This is something positive and with that atmosphere you can have good, positive things for the club."