Clattenburg, who takes charge of Monday's clash between Crystal Palace and Liverpool, live on Sky Sports 1HD, was branded "not of the required standard" by Conte after the Serie A champions crashed out of the Europa League on Thursday.
Clattenburg sent off three players in that game - one Juve, two Benfica - but Gallagher says the criticism will not affect his former colleague ahead of a huge game in the title race at Selhurst Park.
"It's an astonishing remark," Gallagher told Sky Sports. "If he'd said that he'd had a bad game on the night then I can accept that.
"But to say he is not of the required standard is unbelievable and I can't accept that. It's just sour grapes. It's easy to have a go at the referee.
"For me, Mark is in the top-two officials this season with Martin Atkinson. His CV this season has been amazing.
"When he goes into tonight's game he won't worry about that remark because it's not true. He'll go out there and referee like he has done throughout the season."
Meanwhile, Gallagher believes Neil Swarbrick was unlucky to have missed two clear penalties and made a fantastic decision not to award another in the 0-0 draw between Chelsea and Norwich at Stamford Bridge on Saturday.
Norwich's Martin Olsson was brought down in the box early on under challenges from Ashley Cole and John Terry and Gallagher said: "Both Olsson and the referee are unlucky.
"The ref's unlucky with his position. He must think Ashley Cole has won the ball and it's run through to the goalkeeper, but when you break it down John Terry has fouled him and it's a penalty."
Chelsea striker Demba Ba later called for spot-kick after he clashed with a defender off the ball and Eden Hazard was up in arms when he was seemingly brought down in the box, yet replays show no contact was made.
Gallagher said: "Ba's incident is a penalty but it's impossible to see it and impossible to give. Both the referee and the assistant are following play and it's just one of those things you have to accept.
"With Hazard, after seeing it, I immediately thought it was a penalty but after watching the replay it definitely is not. He's overrun the ball and whether he has seen the tackle coming or not, I don't know.
"You have to give full credit to referee there for not giving it. For him to get that right is an amazing piece of judgement."
Watch Crystal Palace v Liverpool on Sky Sports 1 from 19.00 on Monday.