The Belgium international suffered the problem in Saturday's 3-1 win over Everton and is set to miss a number of crucial games in the coming weeks.
Kompany was due to link up with the Belgium squad ahead of Friday's World Cup qualifying game in Croatia, but he has been forced to withdraw.
The 27-year-old, a key player in City's backline, could also miss Premier League games against West Ham, Chelsea and Newcastle, and the Champions League clash in CSKA Moscow.
City suffered a shock defeat in Cardiff while Kompany was sidelined with a groin problem earlier this season and his absence will be a concern to manager Manuel Pellegrini after some erratic defensive displays by his team.
Serbian defender Matija Nastasic replaced Kompany at the weekend and is likely to deputise during his injury lay-off.