Andy Little netted his seventh goal of the season for the Glasgow giants, before Fraser McLaren levelled for the home side, who could have snatched victory at the end.
It was not quite up there with Berwick's famous Scottish Cup win over Rangers in 1967 but they would have been the happier of the two sides with the outcome.
Anestis Argyriou made his Rangers debut from the start after signing a two-year deal on Saturday.
Fellow new recruit Sebastien Faure - who joined the Ibrox side last week - was named on the bench.
Skipper Carlos Bocanegra also started despite being an injury doubt before the game, while Ross Perry made his first appearance of the season after recovering from a calf strain.
Like Old Firm rivals Celtic, Rangers had expressed a desire to play in England in the past.
But the clash at Shielfield, two miles south of the border, was a long way from the promised land of the Barclays Premier League.
Rangers started well and were threatening early on when a Barrie McKay cross found Lewis MacLeod and he fired wide.
Lee Currie then picked up the first booking after just four minutes for a crunching challenge on Ian Black.
Berwick refused to be overawed by their visitors and Dean Hoskins tested Neil Alexander at his near post with a cross come shot.
Neil Janczyk then had a go with an effort from distance that deflected past several bodies in the box before being smothered by the goalkeeper.
The home side continued to push forward in search of the opener and Darren Lavery dragged his shot wide when he should at least have tested Alexander.
Berwick goalkeeper Youssef Bejaoui did not have a save to make for most of the first half but was left picking the ball out of the net in injury time.
A Black free-kick found Lee McCulloch at the back post and he nodded into the path of Little to rifle home from six yards.
Berwick tried to hit back immediately after the restart and Phil Addison saw his shot fizz across goal and wide, before Alexander was called into action to gather from Currie.
They were back on level terms after 63 minutes when substitute McLaren sent a powerful angled drive past the grasp of Alexander and into the bottom corner just four minutes after coming off the bench.
At the other end, Francisco Sandaza's point blank shot was blocked by Bejaoui, before he was withdrawn along with McKay to make way for Kevin Kyle and Robbie Crawford.
Berwick looked like they were going for the win and Marc Lancaster was not too far away with a ferocious shot that whistled just over.
Alexander then produced a superb save to tip over from McLaren with a couple of minutes to go.
Chris Townsley nodded home from the resulting corner but the referee had already blown for a foul.
