Roberto Mancini's side take on Wigan at Wembley on Saturday with the chance of winning their second FA Cup in three seasons.
But first up is Tuesday's Premier League home clash with West Brom, with a win needed to secure a place in next season's Champions League for second-placed City.
Kompany told reporters: "Saturday is a big, big day for the club, it's massive and everyone realises that.
"But it would be nice to win the game against West Brom first. That would set us up really nicely to focus on the final.
"It's really important that we qualify for the Champions League first.
"There is no other way to think about this game.
"Until we are safe for that Champions League spot we have to play the game like every other game."