Both sides go into the Group A game after suffering defeats on Friday night, Scotland losing 2-1 at home to Wales and Serbia going down 2-0 to neighbours and bitter rivals Croatia.
Caldwell said: "Their result would have been a huge blow to them.
"It is probably the equivalent of a Scotland versus England game and if we had lost that it would have been a big blow to us, so we have to try to use that to our advantage.
"We have to start the game a lot better than we did on Friday and get the crowd on their backs.
"It is up to us to keep the ball and frustrate them early on and try to turn the fans against their own team
"We know that, but we have to go out there and do it.
"I think there is enough confidence and ability. It is just a case of going out and showing that on the night and getting it right over 90 minutes, getting a win under our belt and then trying to build on it."