Danny Kedwell's header and two Chris Whelpdale goals, the second from the penalty spot, earned the Gills a vital three points in their fight for a play-off spot while Alan McCormack scored Swindon's reply.
The defeat failed to stop the Robins bouncing back to League One at the first time of asking though as rivals Torquay lost at AFC Wimbledon. With Shrewsbury only managing a draw, Di Canio's side are four points clear at the top with two games left.
"We have to focus on the title now," said the Italian. "The last two performances are due to a long season, but I am happy for the players and the fans because after 10 months we can celebrate promotion.
"Only one team can be called the champions, and I want to be called the champions because I am a greedy man. We have this in our hands. We are four points clear so we need to win on Saturday. I will do everything I can in the week to help them keep their focus."
"It's fair enough for the players to celebrate as they travel back and raise a glass, I will do the same with some red wine. But not to get drunk, I don't see why they should go out until four o'clock in the morning and get drunk."
