Sepp Blatter, the president of world football's governing body, has this week admitted that they may have made "a mistake"in awarding the tournament to the Arab state, where matches would have to be played in temperatures of up to 50 degrees if played in the traditional summer time slot.
The FIFA Executive Committee will vote on switching the World Cup to the cooler winter months this October, but such a move would play havoc with the calendars of the major European leagues.
And Carragher says the best way to solve the problem would be to award the competition to another nation.
"It raised eyebrows when we saw that the World Cup was going there," he told Sky Sports News.
"Is it a 'mistake'? Is that being disrespectful to Qatar? I don't think it's ideal to be playing a World Cup in those conditions.
"So now people are talking about a winter World Cup, but how does that affect the Champions League, the Premier League, La Liga, the Bundesliga and all of the leagues around Europe?
"For me, a World Cup is at the end of a season in the summer. If FIFA have messed up and the conditions aren't right then I think you have to look at it again and give it to somewhere else.
"You can't be messing about with all the leagues in Europe and how it's set up. It's been this way for so long.
"If they've made a mistake I think they've got to rectify it."
When it was suggested FIFA should have been aware that Qatar's climate would prevent a summer tournament taking place before they were voted as 2022 host in 2010, Blatter said: "That may well be so, and it may well be that we made a mistake at the time."
In an interview by insideworldfootball.com, Blatter added: "On the other hand, you must also consider political and geo-political realities. The World Cup is FIFA's biggest, if not only, global event. Who are we, the Europeans, to demand that this event has to cater to the needs of 800 million Europeans above all?
"I think it is high time that Europe starts to understand that we do not rule the world anymore, and that some former European imperial powers can no longer impress their will on to others in faraway places.
"We must accept that football has moved away from being a European and South American sport, it has become the world sport that billions of fans are excitedly following every week, everywhere in the world."
Regarding proposals to switch the event to winter, he said: "After many discussions, deliberations and critical review of the entire matter, I came to the conclusion that playing the World Cup in the heat of Qatar's summer was simply not a responsible thing to do."
The decision to re-think the timing of the tournament has led to anger amongst the European clubs and Leagues as a switch to winter would mean the World Cup being played in the middle of their domestic seasons.
And the Association of the European Professional Football Leagues (EPFL) has called for FIFA's decision into a change of timing to be postponed.
The EPFL said in a statement: "The feasibility of any change to the calendar involves every aspect of football organisation across all countries, from the grass-roots to the major national representative tournaments and the leagues.
"Any suggestion of staging the World Cup at a time other than the traditional period is a crucial issue for the European Leagues considering the impact on the organisation of domestic championships throughout Europe, and of the international football calendar around the world.
"The EPFL is of the view that no hasty decision shall be made by the FIFA executive committee of October 3-4."
The statement adds "such an important decision cannot be rushed with artificial deadlines" and says that a proper medical assessment on the impact of the heat on players and supporters should take place.
The EPFL also want FIFA to look at the consequences a move may have on the transfer system, any effects on commercial and media contracts and possible effects on the amateur game.