Frenchman Yohann Lasimant nodded home his second goal of the season, three minutes after coming off the bench, to give Orient a first league win in five - which moved them back to the top of League One after Brentford had temporarily overtaken them earlier in the day.
Adebayo Akinfenwa put the Gills into a fourth-minute lead with a looping header before Elliot Omozusi saw his spot-kick saved by home keeper Stuart Nelson - not the first time this season the O's have missed a penalty.
But on-loan Orient midfielder Marvin Bartley restored parity with a header 20 minutes from time before Lasimant grabbed the winner after Leon Legge had cleared his initial effort off the line.
"Credit to Yohann, whoever scores, they're all important," said Slade. "We may never know how important his winner is, though.
"The players never give up, and they showed great character in that second half. I told the players to keep playing and keep believing at half-time. That was very, very important.
"We got off to the worst possible start, we did not do our jobs properly, which is unlike us. That gave something for Gillingham to cling on to and they looked the brightest early on.
"We gradually got into the game, we played some really good football between the two boxes.
"Sometimes you think it's not going to be your day, we've missed five or six penalties now, and had a few difficult takers.
"I've looked to see who's got the biggest feet, which is our chief executive, so he's going to come on and take the next one."