New boss Paolo Di Canio, who has taken over the reins from Martin O'Neill, claimed his players were "not the fittest team in the world" following Sunday's 2-1 defeat at Chelsea.
But Gardner told the Northern Echo: "We work hard every day, no matter what manager comes in, we're professional footballers.
"I don't know where this has come from, that Paolo Di Canio will work us harder, because every club you're at, there's only one place you'll be found out, and that is on the pitch.
"There's been people talking and chatting about this, but it's a load of rubbish basically. The gaffer's come in and it's the same.
"Every football club has to work hard. We were working hard, we have to if we want to stay in the Premier League. But the gaffer's been brilliant with all of us since he arrived."
Sunderland are only out of the Premier League's bottom three on goal difference after losing at Stamford Bridge but the prospect of relegation does not concern Gardner.
He said: "Without sounding big-headed, I've not thought about being relegated. I haven't thought about anything like that - all I've thought about is how high we're going to finish in the table.
"We've got ability in abundance, we've got work-rate, and we'll get there. I thought the performance at the weekend showed that, I thought we were brilliant."
Gardner will miss Sunday's Tyne-and-Wear derby against Newcastle because of suspension - he incurred a 10th booking of the season during Sunday's 2-1 defeat at Chelsea for fouling former Magpie Demba Ba, who he feels was guilty of play-acting.
Gardner added: "It's not the start that I wanted, but it's not a yellow card. It was a fair tackle. He's just rolled around and forced the booking."