Gordon Strachan's side have been handed a tough start to their Group D campaign with a showdown against the top seeds away from home.
And with Poland, Georgia, Republic of Ireland and Gibraltar also in the group, Berra believes a positive result in Germany could prove significant later on.
Berra said: "If we are going to get something there then we will need the type of performance which the players produced under Alex McLeish to beat France in Paris.
"We will be massive underdogs but in many aspects it is a great way to start. If we get something from Germany you would almost look upon it as a bonus point.
"The manager has already talked about the Germans always qualifying for major finals and it is not being pessimistic to say we are fighting for second place."
Berra also thinks that Strachan's side can approach the game with little pressure, as they will not be expected to win.
He added: "We'll go into the group confident and thinking we can win all of the games. That's the way we approach things under the manager.
"Germany are one of the elite nations and had nine wins out of 10 on their way to qualifying for the World Cup in Brazil.
"For them to lose against Scotland at home would probably be ranked as an embarrassment given the level they are operating at so we can go there and look to gain.
"We'll put on a show and be competitive"