Ally McCoist's side were twice hauled back having edged ahead in the contest before breaking free in the final quarter of the game.
The Light Blues took a sixth-minute lead when David Templeton got the better of his marker and played the ball to Andrew Little, who crossed for Robbie Crawford to score.
The home side refused to buckle though and went close to an equaliser after Steven Jackson powered forward only to shoot wide.
But they were back on level terms after 21 minutes, Paul Quinn drilling a penalty down the middle of the goal after Andy Mitchell was adjudged to have fouled Kevin Turner in the area.
Having worked hard to get back into the game, The Shire were then reduced to 10 men after Turner was dismissed for a crude challenge on Ian Black.
The red card appeared to offer Rangers a boost and they went close to scoring a second two minutes before the break courtesy of Kal Naismith, but he curled his left-foot shot just over the crossbar.
Templeton ensured they would go into the break in the lead though after curling home a delightful free-kick two minutes into time added on after he had been fouled on the edge of the area.
But East Stirling showed their self-belief and drew level for a second time in the 50th minute, Quinn on target again as he headed home unmarked in the box.
A topsy-turvy game then took another twist in the 64th minute, Black giving Rangers the lead for the third time.
The midfielder played a one-two with Little before shooting across Shire keeper Grant Hay and into the bottom corner.
Templeton then helped himself to a second goal with 11 minutes remaining, the former Hearts winger lashing home inside the area following some good work from Danny Stoney.
There was then a competitive debut for youngster Andy Murdoch late on with the points now in the bag for Rangers.