Lewis Macleod and Kyle Hutton also claimed a goal apiece against Elgin, while Daniel Moore and Mark Nicolson found the net for the home side in an entertaining top-of-the-table tussle at Borough Briggs.
Rangers made a quick start and took the lead after five minutes when Little collected a neat pass through the middle from David Templeton before rounding goalkeeper John Gibson and slotting into the back of the net.
Elgin were back on level terms with 17 minutes on the clock when the ball broke for Moore and he sent an impressive curling effort past the reach of Neil Alexander.
Rangers restored their lead after 24 minutes when Little grabbed his second of the day.
Templeton was again the provider with the cut-back and all that was required was a simple tap-in from the Northern Ireland international.
Again, the home side responded positively with a goal of their own.
Chris Hegarty was booked for a foul on David Niven on the edge of the box and Nicolson dispatched a superb swerving free-kick into the top corner with 32 minutes gone.
Elgin could have added to their tally when Brian Cameron unleashed a decent strike but Alexander was able to smother.
Instead, it was Rangers who had the net bulging again four minues before the break.
Little collected from Hutton before supplying the cross for McCulloch to lash home in off the crossbar.
Rangers made a change at the break when Anestis Argyriou was replaced by Ross Perry.
McCulloch - back in a forward role after a recent spell in defence - was on target again when he raced onto a Templeton ball and fired across the goalkeeper into the bottom corner after 60 minutes.
Macleod added his name to the scoresheet four minutes later with a powerful, low strike from just inside the box.
Hutton put the gloss on the win when he rifled home from 20 yards after 83 minutes, before Alexander denied Moore from the penalty spot two minutes later after Hutton brought down Stuart Leslie.