Peter Herbert's organisation asked the Metropolitan Police to investigate whether there was a racially-aggravated element to the "inappropriate language" Clattenburg allegedly used to Blues midfielder John Obi Mikel during last month's game against Manchester United.
Chelsea lodged an official complaint with the Football Association, but neither the club nor Mikel went to the police with their claims and the investigation has now been dropped.
Herbert told Sky Sports News: "What seems to have happened is that although Chelsea made a report to the FA they've decided, in their infinite wisdom, not to submit the same papers to police.
"It appears on the face of it to be another example of the football industry seeking to sweep racism under the carpet.
"If there is anything we can do to put pressure on the victims concerned to go forward to the police with this matter, then we'll do that.
"It highlights the fact that football can not regulate itself and really, on this issue of race, always wants to take the path of least resistance."
Herbert also told the BBC: "It sounds remarkably like a football cover-up...
"We strongly suspect that the FA and/or Chelsea have failed to provide this information to the Metropolitan Police in order for them to conduct a proper investigation.
"The information we had is that there are 'no victims'. Well, if there are no victims, what on earth has been referred to the FA in the first place?
"What on earth are the FA and Chelsea playing at then? Are they having some cosy exchange of statements between themselves and not giving it to the police?
"We're going to ask the borough commander for an explanation. Was there any co-operation? Was any evidence given? If none was given by the FA or Chelsea we want to raise that issue with the Minister of Sport."
Herbert believes a thorough police investigation would also be in the best interests of Clattenburg, who has not been charged with any offence and strongly denies any wrongdoing.
He said: "Given the publicity this engendered it's fair for all concerned, including the person who's alleged to have made the comment, that justice is seen to be done and it's not dealt with by the FA internally."
But a Chelsea official later told Sky Sports News the FA is the appropriate body to deal with the complaint, and its decision on whether to charge Clattenburg is expected by the end of the week.