Pellegrini coached the Argentine at Malaga before leaving this summer to replace Roberto Mancini at the Etihad Stadium.
And Caballero has now revealed that Pellegrini tried to persuade him to follow him to Manchester City.
However, the 31-year-old says he turned down the opportunity because he did not know whether he would play ahead of Joe Hart, and last week he signed a new deal to stay at Malaga until 2017.
"I talked with Manuel Pellegrini before he left Malaga," Caballero said, quoted in The Sun.
"I knew of his intention to take me with him to Manchester City.
"But I was unsure whether he would stick with the same keepers he had already."