Arsene Wenger's side were handed a reality check on Tuesday night when a late goal from Robert Lewandowski gave Borussia Dortmund victory in the Champions League.
The manager felt they had been "naive" to be caught on the counter attack when they should have been concentrating on closing out a positive result.
Arsenal's next opponents, managerless Crystal Palace, will not provide anywhere near the same level of challenge at Selhurst Park - but Mertesacker insists discipline is needed nonetheless.
"We know we are on a good run in the Premier League, but it needs more than just playing with full freedom and vision," said the German.
Breathtaking goals in Arsenal's 4-1 dismantling of Norwich last weekend won widespread plaudits, but Mertesacker added: "We need to defend first.
"Then we can switch the rest on. You have seen what can happen after that. We have players who can play so quickly together.
"Tomas Rosicky came in and fitted into everything as well as Mesut Ozil or Jack Wilshere. When those three combined together, you could feel some magic in the air, a real chemistry."
Ozil was subdued against Dortmund, which was a first for the German playmaker since his club-record £43million deadline day move from Real Madrid.
The midfielder, though, almost helped conjure up a goal for Santi Cazorla, who curled a 20-yard effort against the angle.
Cazorla, just back from an injury lay-off, is excited by the prospect of linking with such a high-quality team-mate.
"We have got to know each other and I have to say that it is very easy to have a good understanding with a great player like him, since our playing philosophy is very similar," Cazorla said.
"As the days go by we have a better association on the pitch. Mesut is a player of a different kind. He is a joy to watch and has a lot of talent. Like Zidane or Ronaldinho, he can be one of those players who marks an era."
Arsenal will check on the fitness of Wilshere, who was substituted as a precaution against Dortmund after aggravating his ankle, before the Palace game.
Mathieu Flamini should be back in contention having been rested in midweek after suffering mild concussion against Norwich.