The Canaries are third from bottom after a disappointing start to the season and face a daunting trip to the Premier League leaders.
Olsson was part of the Blackburn side that pulled off a shock FA Cup fifth-round win at the Emirates last season - and he insists the Canaries are capable of a similar result.
But the Swedish midfielder admits his struggling side must hold their nerve when they inevitably come under pressure.
"We have nothing to lose," said Olsson.
"Everyone thinks we're going to lose to Arsenal. We've just got to turn it around and hopefully get three points.
"We'll just go out and play our game and see what happens.
"We sat back and were hoping they weren't going to score too many goals.
"We got our chance at the end and we defended well as a team. That's what we have to do now as well.
"Patience. They're going to put pressure on us, create loads of chances.
"We've just got to keep calm and hopefully catch them on the counter attack."