The Gunners finished last season with a flourish, allowing them to secure a top-four standing and a place in the UEFA Champions League.
The challenge now is to build on those efforts and challenge for major titles.
It is well-documented that Arsenal have gone eight years without landing silverware, but Mertesacker is convinced that Arsene Wenger is piecing together a side capable of ending that barren run.
The Germany international defender said in The Observer: "We must be well prepared both mentally and physically to do it straight from the off.
"It's very hard to be consistent and that's the main reason for Manchester City and Manchester United being successful in the last two years. They have shown for the whole season that they are consistent and that will be our main focus for the next season.
"We haven't sold any players yet so we are coming from a good base. We're at a good level and no-one is leaving.
"Maybe someone will come in and strengthen our squad, so it's a better position to be in than last summer when we lost two important players (Robin van Persie and Alex Song).
"I would say consistency is the most important thing. If we can do that, then we should be challenging for trophies and titles."
Mertesacker added: "It's a big year for us. We've got the World Cup at the end of it but the focus has to be on Arsenal because we want to win things.
"We had such a strong finish at the end of last season, especially in the final 10 games, and that is the target for the whole season."