Allen led the Gills to the League Two title last season but departs after winning just two of 11 league games this term.
Saturday's 2-0 defeat to Shrewsbury left the club only a point above the League One drop zone but Scally told Sky Sports News that Sunday's move to relieve the 48-year-old of his duties was a considered - and difficult - one.
"The decision was not down to Saturday's result. It was a decision that was pending and it was made for good reason," Scally said.
"It was a unanimous decision between three board members that the time was right and I fully support it.
"I think if you look at the last quarter of last season and the first quarter of this season, our win record has been very poor and I didn't see any signs of that changing.
"I could have waited until Christmas, I could have waited until January or February, and maybe things would have changed, but - and I have watched football for a long time - I didn't see any change in the way we were playing, I didn't see the players playing to their potential.
"Two wins out of 13 - because we were knocked out of two cups - was a concern and the way we've been playing with the quality of the players we've got was a concern. If we'd continued on current form, we'd be relegated, without a shadow of doubt.
"I have to say the decision I made on Sunday was probably the most difficult I've made in football because I like Martin Allen very much and of course he did win us the League Two championship last year.
"But the longer I left it, the more difficult I felt it would be to salvage a place in League One next season and that is key to the future of this football club."
Gills chiefs will now turn their attention to Allen's successor, with Peter Taylor an early frontrunner, but Scally says a replacement has not yet been identified.
"We've haven't got anyone lined up," he added. "John Schofield (assistant manager) will run the team this week and is quite capable.
"We haven't got anyone in mind yet but, as you'd expect, we've had numerous applications and CVs coming in."