Gills have won only one of their last seven League Two matches at Priestfield, losing four times during that sequence after going down 1-0 to improving Wycombe on Monday night.
The run has seen the Kent club surrender top spot in the table to Port Vale - one of the teams to win at Priestfield recently.
"We are finding it difficult. Let's not hide, we are finding it difficult to get the wins at home and playing for 95 minutes," said Allen, whose side were beaten by an 85th-minute goal from Matt McClure. "That's something we need to deal with.
"In the second half we didn't play well enough, we were far too open. On the counter-attack they looked very dangerous and it was a disappointing second half all-round.
"They have pace up front and good attacking players. We never got on top of them. They played well second half, but we didn't and that's something I have to address.
"I think we have one of the best defensive records in the country but after this you would think that statement was crazy."
