The hosts had lost their last five league outings but picked up their second win of the campaign after Brian Allison grabbed the only goal of the game.
Having struggled on the road so far this season, this was the match which appeared destined to provide Rangers with their elusive first away win in the league.
Ally McCoist's men faced a side rooted to the bottom of the table, whose manager Greig McDonald was absent from the dugout to attend his own wedding, and knowing victory would propel them to the Third Division summit.
They were also boosted by the return from injury of key players Dean Shiels and Ian Black, who both started at Forthbank.
Instead, it was Stirling who surged into the lead after just eight minutes.
Mark Ferry delivered a corner to the back post which dropped for Allison, who took advantage of dreadful defending from Rangers to bundle over the line.
The visitors could have hit back through a set-piece of their own when Black squared a free-kick to Lee McCulloch on the edge of the box and he dragged a low shot just wide of target.
The skipper then rattled the woodwork when he met a Fraser Aird cross from the left only to see his bullet header smack off the inside of the post.
Aird had a go himself with a well-struck shot that fizzed across the face of goal and past the post, before Jordan White failed to add to the home side's tally with a similar effort at the other end.
Rangers cranked up the pressure with half-time approaching and McCulloch stung the palms of goalkeeper Sam Filler with a powerful free-kick, with an off-target Black also failing to find the leveller just before the break.
The Binos made a change at the interval when Filler was replaced by substitute keeper Mark Peat.
Peat was immediately called into action to produce a decent reflex save to prevent a sidefoot effort from Shiels finding the back of the net.
Marc McCulloch then did well to hook a header from Sebastien Faure off the line as the Glasgow giants piled forward in search of an equaliser.
Rangers could have had a penalty when Shiels went to ground under pressure from Kieran McAnespie but there was no spot-kick from referee Bobby Madden.
Kevin Kyle was thrown into the action with 20 minutes to go and nodded just over from a Lee Wallace cross, before Steven Weir squandered a great chance for Stirling.
With time fast running out for Rangers, Shiels delivered an inviting ball across goal for Kyle but the big striker failed to connect in front of the gaping goal.
Shiels then stabbed wide in the final minute as Albion held on for the win.