The 27-year-old Frenchman, appointed by boss Alan Pardew after an injury to regular skipper Fabricio Coloccini, was outstanding in the 2-1 victory over Stoke on Sunday.
He scored the equaliser with a superb free-kick, and Papiss Cisse scored the winner deep into stoppage time.
Cabaye insists he is not equipped to deliver impassioned speeches in the dressing room and prefers to lead by his actions on the pitch.
"The manager trusted me and the club trusted me, and I want to give them back this confidence on the pitch," he said.
"For me, it's a new role. My English is not very good to speak in the dressing room, I just want to lead the team on the pitch and I hope I am doing well."
Cabaye's free-kick was of the highest order, getting the ball up and over the wall from the edge of the box and past Asmir Begovic.
Whilst Andre Marriner sorted out some pushing and shoving in the box, he rolled the ball back a foot to give himself an extra yard of space to play with.
Marriner spotted his cheeky attempt and ordered him to put the ball back to its original spot - but it made no difference, the midfielder finding the top right hand corner.
Cabaye said: "I wanted to move the ball back, but the referee didn't want me to.
"I just wanted to hit the target. The wall was so tall and I couldn't put any pace on my shot. I just wanted to hit the target.
"I love this kind of free-kick and I practise them a lot on the training ground, so I am very happy."
"It's a fantastic result because it was a very difficult game, a very tough game. Stoke came here with a nice shape, a nice unit, 4-4-2, and it was very difficult for us to play. But we won the game at the end and for us, it was fantastic."