Chris Whelpdale opened the scoring on 24 minutes and turned provider for Gillingham's second shortly after as Danny Kedwell fired in his sixth goal of the season.
Callum McGregor then hit a brilliant consolation for the visitors, but it was the Gills who picked up their first win since Martin Allen was sacked earlier this month.
McGregor's solo effort from 25 yards, 11 minutes from time, offered Kiwomya little comfort.
"I think we've been beaten by ourselves," said Kiwomya.
"We were the better team by far, but when you give away goals like that, you give yourselves a mountain to climb. It's not even one goal, it's two.
"The first goal we should have headed it and stopped a cross, the second goal they pass it over our heads.
"Gillingham hardly did anything to deserve that, but they led because of our mistakes. It's about players doing things right - heading it, kicking it, dealing with their man.
"They're the simplistic things we have to get back to doing. We're good at football, but we need to win games.
"I spoke to Callum McGregor about getting into those areas and getting in shots. It was a brilliant goal.
"I knew we'd get back in the game at some point, it was just about getting that second goal. I thought we did enough in the game to win the game, it's disappointing."