Rabbatts was responding after a Sunday newspaper disclosed private emails sent by Scudamore that contained inappropriate remarks. He has since apologised.
A Premier League Commission will meet next week to discuss the emails.
Rabbatts issued a statement which read: "No one can doubt the tremendous achievements of the Premier League in creating one of the world's great footballing competitions.
"But with that success and the massive public interest it generates comes the obligation to behave responsibly and have in place proper lines of accountability and good governance.
"Sadly recent events appear to show that these things are currently lacking in the administration of the Premier League and indeed there is growing evidence of a closed culture of sexism, symbolised in the email exchanges which have been made public.
"It is increasingly clear that steps are needed as a matter of urgency to review governance at the Premier League with a view to improving accountability and tackling head on a culture that demeans women and seems to discourage their involvement in the game's administration.
"These challenges go beyond the current situation of chief executive Richard Scudamore, however if the League are to move forward in a positive way then he and they should give serious consideration to his position in the coming days.
"I personally hope that progress can be made on all of these fronts so that we can feel confident that the leaders of football are accountable for their actions and support a culture that genuinely welcomes the participation of women and girls in our national game."