Trouble began to surface in the second half of the contest and Horne has promised the FA will do all they can to help identify those involved and will look for criminal charges to be brought.
"The FA deplores scenes which have taken place. They are unacceptable and everything will be done to take action against those involved," he said.
"The Metropolitan Police and the FA are this evening investigating scenes of sporadic violence and disorder in the Millwall FC end at today's FA Cup Semi-Final.
"The FA and Wembley Stadium will work with police and representatives of Millwall FC to review all events.
"We will look to ensure those involved are identified and we would call for criminal charges and a football banning order to be brought against them."
Millwall CEO Andy Ambler also took a hard line on the matter, promising they would act decisively against those involved.
"Our position is clear. Anyone associated with our club found guilty of violent behaviour will be banned indefinitely from Millwall matches in addition to any punishment they receive from the authorities," he said.
"Having worked so hard to show the positive side of our club both on and off the field, we cannot allow the actions of a mindless minority to undermine that.
"At this stage we are still in the process of establishing the full details of what happened in a section of the ground during the second half.
"When we have those facts at our disposal and the police have completed their investigations we may be in a position to comment further."