The Manchester City playmaker has played in two previous cup finals at Wembley and finished on the losing side on both occasions.
Nasri was part of the Arsenal side stunned by Birmingham City in the 2011 Carling Cup decider, and suffered a similar fate last season when Wigan defeated City in the FA Cup final.
City are overwhelming favourites to beat Sunderland on Sunday but Nasri is taking nothing for granted.
"At the moment Wembley has unhappy memories for me," said the France international.
"After losing two finals there, it is time for me to win one. They were similar games and I want to avoid the same thing happening.
"We were heavy favourites to win both finals, just as we are this time. The memories still hurt, and what is worse is that at the end of the season the teams that beat us went down.
"You go back home after those games, play the game again in your head and it's difficult to handle."
Nasri's career has been rejuvenated under Manuel Pellegrini this season and the midfielder would love to win the trophy for the City boss.
The 26-year-old said: "From the very first training session he told me exactly what he wanted from me.
"I really like the way he talks to me and he made me feel important again, and after that I go out on the pitch and try everything for him.
"Because he trusts you so, you have to do everything for him."
Watch Manchester City v Sunderland live on Sky Sports 1 from 12.30pm on Sunday.