The move "follows a conversation in which discriminatory abusive comments were made to Richard Rufus," according to a statement on the FA's official website.
Elliott, who played in defence for a number of clubs including Charlton, Pisa, Celtic and Chelsea during his playing career, offered his resignation and the FA has accepted it.
The 48-year-old was a member of the Association's Judicial Panel and was a nominated member of UEFA committees. He has also left his role at Kick It Out.
"I wish to thank Paul for his dedicated and unstinting work, particularly in the area of anti-racism," FA Chairman David Bernstein told their official website.
"I am saddened by this turn of events and it is with regret that we accept Paul's resignation.
"However, the use of discriminatory language is unacceptable regardless of its context and in effect has made Paul's position untenable."
Elliott became the first black footballer to receive the CBE for services to equality and diversity in football earlier in February.