The Magpies responded positively to the sacking of Chris Kiwomya to record just their third league win of the campaign and move off the bottom of League One.
Danny Haynes gave County the lead after 19 minutes at Meadow Lane on Tuesday night, only for Adam Rooney to restore parity on the stroke of half-time.
Yoann Arquin put the hosts back in control on the hour mark before Callum McGregor made it 3-1 with his eighth goal of the season in the 68th minute.
Oldham striker Jonson Clarke-Harris' instant reply ensured a nervy finish for County but they clung on for a crucial win.
"This is just one win, we need to string together a few wins now and get out of this situation," said Hodge.
The 51-year-old deflected praise away from himself and onto his players. He was particularly impressed with how County's younger players fared after they were thrown in at the deep end.
"We have to stick together and they did that and ground out this victory," added Hodge.
"It's great for the young players to get a victory. Young Callum (McGregor) is a technically excellent player and he will go on to bigger and better things.
"I have had to change my team sheet four times, I have lost five players before the game with either injury or illness.
"You never know whether a young player is good enough until you throw them in, and Jordan Holt did very well on his debut, he always shows a great calmness in his game and he did that.
"We gave ourselves a tough 20 minutes conceding the second goal, but we dug in and got ourselves a terrific result."