The French forward scored his side's second goal in the 2-0 win over Leeds United to set up a last-16 meeting with Manchester City.
Gouffran admits the French version of the League Cup is not taken seriously in his homeland, but he says everyone in the Newcastle squad see the competition as the chance to win some silverware.
"In France the League Cup is not well thought of by fans or the teams or the players," Gouffran told the Evening Chronicle.
"Often a reserve team is put out in these competitions.
"Here, though, with the way the league is we want to get out there and try to win some silverware.
"That is what we will try and do. We want to get as far as we can."
Meanwhile, Gouffran hopes his goal against Leeds will boost his chances of earning a starting place in Monday's clash against Everton.
"That is why I am here - I came to play," said Gouffran.
"Last season I started every match. This season it has been a bit more difficult but I am not giving up.
"I will go out and give 100% so I can play the next game."