Junior Hoilett saw his penalty saved by Tim Howard a minute after half-time before Mauro Formica fired Rovers' second spot-kick against the post.
Arteta then delivered the killer blow in injury time, after Christopher Samba was judged by referee Lee Mason to have climbed on the back of Marouane Fellaini, to hand Everton their first league win of the season.
Defeat left Steve Kean's side pointless from three games as they somehow threw away victory in Brett Emerton's final game for the club.
The Australian was handed the skipper's armband after announcing in midweek he was returning to his homeland to join Sydney FC after eight years at the club.
The 32-year-old was one of three changes made by Kean, with Samba and Gael Givet making their first league starts to strengthen the home defence.
Everton also welcomed back key men with Arteta and Fellaini joined in the starting XI by Victor Anichebe as Tim Cahill, without a league goal in 2011, dropped to the bench.
Despite both sides still looking for their first point of the new campaign there was no sign of early caution and Rovers could have had an early opener but for the woodwork.
David Dunn latched onto Hoilett's pull-back in the area and after losing his marker hit a low angled shot that rebounded back off the post.
Jason Roberts was then sent sprawling in the area by Howard but referee Mason correctly adjudged the Everton goalkeeper had got a touch on the ball first.
At the other end Leon Osman headed over but after a bright start the game started to become scrappy as both teams gave away possession too easily.
Michel Salgado played an easy pass straight out, while Osman and Jonny Heitinga shot wide when well placed.
Injury forced Kean into a double substitution after 34 minutes with Morten Gamst Pedersen and Dunn going off.
Summer signing Radosav Petrovic and Formica replaced them.
Salgado made a timely headed clearance just before the break with the taller Fellaini seemingly better placed.
Blackburn won their first penalty a minute after half-time when Ross Barkley brought down Formica in the area.
The Everton teenager had lost the ball in midfield and after trying to make up for the error he left a leg out that the substitute fell over after checking inside.
But Howard, whose blunder cost the Toffees a 1-0 opening-day defeat on the same ground last year, saved Hoilett's spot-kick with a diving save to his left.
Despite the miss Blackburn, and especially Hoilett, stepped up the pressure and Howard had to save again after a Samba header crept through a host of bodies.
Martin Olsson fizzed a free-kick just over before Formica shot when Emerton was better placed to his right.
Everton boss David Moyes needed to make a change and hauled off Barkley for the experience of Cahill.
The visitors then began to get back into the game and Cahill's lay-off allowed Leighton Baines to cut inside and force a good save from Paul Robinson.
Fellaini then wastefully side-footed over when he ran onto Anichebe's low centre just inside the area.
A slick Blackburn move ended with substitute David Goodwillie pushing a shot onto the crossbar, although he was flagged offside.
Howard then made his second telling contribution 12 minutes from the end when he clawed away Formica's close-range shot.
Goodwillie had beaten Baines and cut a pass inside but with the Argentinian seemingly set to score Howard blocked with an out-stretched arm.
Blackburn's second penalty miss arrived on 83 minutes after Formica had been felled by Phil Jagielka in the area.
Formica stepped up to take the spot-kick but fired it against the upright.
Rovers were then made to pay for their profligacy as Everton were awarded a spot-kick of their own in injury time when Samba climbed over Fellaini.
Referee Mason pointed straight to the spot before Arteta fired in the late winner to the delight of the away fans behind the goal.