Serge Gnabry and Aaron Ramsey struck as the Gunners notched an eighth league win in a row, with Ben Davies pulling a late goal back for the Swans.
The result lifted Arsenal two points clear at the top of the table after Tottenham's earlier draw with Chelsea, on a day when Manchester United and Manchester City both lost.
And the victory was a welcome anniversary present for Arsenal manager Wenger, who was marking 17 years in charge.
"There is nothing better than winning," the Frenchman told Sky Sports.
"The first half was quite difficult, we were slow in our combinations and Swansea were sharper than us.
"It was a question of being patient without making mistakes. After we were much stronger and playing at our level.
"We took advantage of the chances we have created. We have substance in the team and we could see that today in the second half. We came out with purpose in our game and that was important."
Arsenal have now won five games on the bounce since losing on the opening day, and, asked if his team could go on and challenge for a league title that the club last won in 2004, Wenger said: "It is early in the season but we had an opportunity today.
"The sign of quality in a team is that when you have an opportunity you take it. Today we took it.
"Our spirit has been tested after the first home game of the season but the way we have responded is very positive.
"In this job, it is very important that you keep focused and you have a purpose and desire to improve. Let's focus on that."
Wenger also paid tribute to another wonderful performance from Ramsey, whose goal took his tally for the season to eight in all competitions.
"He was like the team today," said Wenger. "It was a difficult start but he got stronger and stronger and had an outstanding second half."