The Black Cats went down 1-0 to the Gunners at the Stadium of Light after Santi Cazorla struck in the 35th minute.
They were then unable to find an equaliser in the second half despite creating a host of chances and playing against 10 men after Carl Jenkinson was sent off in the 62nd minute.
O'Neill said: "It was a great game from start to finish. The tempo was very high early on and it remained so.
"Obviously Arsenal, a top-quality team, caused us a lot of problems in the first half but I also thought we looked dangerous as well.
"But in the second half, we just came roaring at them, and that was even before the man had been sent off.
"Seriously, looking back at a couple of the chances, I just don't know how we didn't score."
O'Neill also lost full-back Danny Rose in the second half with a recurrence of his hamstring problem, while skipper Lee Cattermole lasted only 45 minutes on his return from a knee injury, although his premature departure was due in part to an early yellow card.