Sergio Aguero's fine individual effort on 78 minutes earned City a 2-1 victory at Old Trafford which cut United's lead at the top of the table to 12 points with seven games left to play.
Though the Red Devils are still overwhelming favourites to wrap up a 20th title, Kompany wants City to finish this season positively to take momentum into the next campaign.
"Every game, we have to play to win," Kompany told Sky Sports 1. "I do believe that the way we finish this season is going to have an impact on next season.
"You're always going to start from the beginning and it's a good game to have under our belt to take into next season."
Milner, who opened the scoring early in the second half, also wants City to build momentum, with Roberto Mancini's side facing Chelsea in the FA Cup semi-final at Wembley on Sunday.
The midfielder added: "Regardless of where you are in the table, the derby is a derby and to beat your rivals is massively important. We came here, we worked hard and got the win.
"We've got good momentum going into the weekend."
Kompany was unfortunate to concede an own goal which drew United level and the City captain feels the reigning champions deserved to take the points.
"The derby is always a special game and for us to win this game is massive," said the Belgian. "Our fans are still celebrating and it's a great night for us.
"I think we all worked very hard. They played very offensively and have an unbelievable strike-force. We had to close the gaps and had to be tight to them. We did that.
"I don't think we gave a lot of chances away and if you look at the overall play I thought we were the better team, and we're very happy."