Seamus Coleman opened the scoring in the 19th minute with a header from Gareth Barry’s pass and Steven Naismith tucked away a second just before half-time following great play by Romelu Lukaku, although replays showed the goal should have been chalked off for offside.
Everton continued to look comfortable until deep in the second half, but then with seven minutes remaining Aaron Ramsey tapped in from close range from Santi Cazorla’s cross.
Olivier Giroud, who only came on at half-time for Alexis Sanchez, then headed in to salvage an unlikely point for Arsenal in the 90th minute.
Arsenal had the better of the opening 15 minutes, but it quickly became apparent they were lacking a clinical edge.
Ramsey tried to carve open the home defence with a series of cutting passes, and there were plenty of fancy touches from Sanchez, who harassed the home defence with his pace, but Tim Howard remained untested in the Everton goal.
The only chance Arsenal mustered in the first half came when Coleman cleared straight to Alex Oxlade-Chamberlain. Luckily for the hosts, the England winger side-footed wide.
It took just a couple of minutes for Coleman to get back into Roberto Martinez's good books.
The Republic of Ireland full-back climbed high at the back post to nod Barry's delicate chip past Wojciech Szczesny and give Everton the lead.
Lukaku played a part in the goal, selflessly ducking to give Coleman a clear header, and Mesut Ozil gave Everton a helping hand with some sloppy marking.
Arsenal were rattled, and they almost went 2-0 down moments later.
Naismith looped a header over Calum Chambers and straight into the path of Kevin Mirallas, but his shot trickled inches wide.
Chambers then carelessly clattered into Mirallas, giving Everton a free-kick on the edge of the box.
Mirallas bent the resulting free-kick up and over the wall, and the Everton fans rose to their feet thinking the Belgian had scored, but the ball had instead rippled the side-netting.
Arsenal manager Arsene Wenger vented his anger at referee Kevin Friend for booking Per Mertesacker after a foul on Naismith.
And the Frenchman's mood darkened just before the break when Everton doubled their lead. Lukaku shrugged off Mertesacker inside the Everton half and hurdled the sliding challenge of Chambers before sprinting at the Arsenal defence on the break. The Belgian cleverly cut inside from the right flank and flicked the ball through to Naismith, who beat Szczesny from close range.
It was a beautiful goal, but one that should have been ruled out for offside, replays showed.
Wenger had seen enough of Sanchez and replaced him with Giroud at half-time, but Arsenal's misery continued.
Giroud managed to blast a volley over from close range just 58 seconds after his introduction.
Jack Wilshere was lucky to escape with a yellow card for a late sliding challenge on Barry. Mathieu Debuchy followed him into the book with a petty challenge on Naismith.
Arsenal dominated possession, but they still did not look like scoring.
Howard made his first save of the match in the 68th minute from Giroud, who rolled a weak shot into the American's hands from 12 yards.
Wenger looked to inject some life into his side by replacing Wilshere and Oxlade-Chamberlain with Cazorla and Joel Campbell.
Arsenal finally found the net with seven minutes to go. Cazorla flashed a low ball across the box in front of six Everton players, and Ramsey prodded home.
A cagey atmosphere descended upon Goodison Park, and the home fans breathed a huge sigh of relief as Giroud wasted a good chance to level with a free header.
The Frenchman had the last laugh, though, stunning the home crowd into silence with a last-minute header from Nacho Monreal's cross.