The Saints paid Bologna £12million for the Uruguayan forward, who after a bright start has failed to make a massive impact in the second half of the season.
But Pochettino insists that the price-tag was never an issue, and it was more about getting used to a new country.
Pochettino said: "The first season for any player is tough.
"I want to think it is just him struggling to adapt to a new country and language. I'm sure it has nothing to do with his price."
The Argentine boss is hopeful that Ramirez will not get more than a three-match ban for red card he picked up against West Brom at the weekend.
He added: "I hope it's not because that's our season over."