Lee Hughes claimed the winner on his debut for Vale, although the final touch may have come off home defender Adam Barrett.
Hughes' goal made it 2-0 in the 20th minute following a rip-roaring start from Vale at Priestfield, with Tom Pope having headed in his 24th league goal of the season early on.
Danny Kedwell pulled one back less than 60 seconds after Hughes' finish, but the Gills were unable to force an equaliser.
"We created numerous chances but you have to say you don't want to be 2-0 down in the first place," Allen said.
"At 2-0 down it's tough for the players to come back and play. They could have hidden, but they came back and gave everything they've got.
"I fully expected us to win in the second half to be honest, we had numerous chances but it wasn't to be.
"When you haven't won at home for a few weeks there's always going to be little things slipping into the players' minds. You could say those demons do strange things to their confidence and those two early goals put everybody on the back foot.
"I don't care about the pressure of being top of the league. We've been there for all of the season, and now we've got a challenge on our hands to get past Port Vale."