The Seagulls are regularly the subject of taunts with the city of Brighton known for having a large gay community.
The Brighton and Hove Supporters Club (BHASC) and the Gay Football Supporters' Network (GFSN) have compiled a dossier detailing the abuse, which they submitted to the FA on Tuesday.
A joint statement from the BHASC and GFSN read: "It wouldn't be described as 'banter' if the taunts and chants were about skin colour and something would have been done by now to stop it.
"Brighton & Hove Albion Supporters' Club has spent over 15 years trying to get the authorities to take this regular abuse of one club's fans seriously.
"We would like our fans to be able to go to games with kids or their grandparents and not hear much of the stuff we are subjected to - it's our basic human right.
"Our campaigning with the help of the Football Supporters' Federation has led to 'homophobic abuse' being included in the list of unacceptable behaviour in ground regulations, yet many clubs, their stewards and their fans do not seem to know what constitutes 'homophobic abuse'."