Rudi Garcia's side had seen their perfect run come to an end last weekend with a draw at Torino but this game against their lowly visitors appeared to offer a great chance to immediately return to winning ways.
An own goal by Alessandro Longhi after 19 minutes put the hosts ahead and they were completely dominant.
However, they failed to capitalise on that and were made to pay in injury time as they conceded for just the third time this season.
While they remained top, it meant their lead could be cut to one point with Napoli and Juventus, second and third respectively, meeting later on Sunday.
AC Milan's worst start to a season since 1981 continued as they were held to a 0-0 draw at Chievo which further increases the pressure on coach Massimiliano Allegri.
It could have been even worse for the Rossoneri as Chievo saw a goal disallowed for offside midway through the second half, although Robinho hit the post on a frustrating afternoon for Milan at the Stadio Marc'Antonio Bentegodi.
Both sides ended the game with 10 men, Riccardo Montolivo followed down the tunnel by Sergio Pellissier towards the end of the match.
Balde Keita made his entrance into Serie A with an inspiring performance, and a goal, although it was not enough for Lazio to take all three points as they recorded a 1-1 draw at Parma.
The 18-year-old gave the Biancoceleste the lead early in the second half after tearing the Parma defence apart in the first 45 minutes, but an equaliser from Alessandro Lucarelli soon after earned the home team a point.
The point may not be enough for Lazio's under-pressure coach Vladimir Petkovic, however, with his side still languishing behind the top six.
Marko Livaja grabbed a late winner as Atalanta saw off the challenge of Bologna at the Stadio Atleti Azzurri d'Italia.
The Croatian forward scored with practically the last kick of the game to seal a 2-1 win for Stefano Colantuono's men after Davide Brivio's goal had been wiped out immediately by Rolando Bianchi.
The win lifts the Nerazzurri three points clear of AC Milan in mid-table and leaves Bologna just a point above the drop zone on Sunday night.
A late goal from Daniele Conti saw Cagliari snatch a 2-1 win at home to Torino and end a run of three successive defeats in Serie A.
The midfielder had also given the hosts a first-half lead but a draw seemed the most likely result following Ciro Immobile's equaliser, until Conti struck again two minutes from time.
It was only the second win in 11 matches for Cagliari but lifts them above Torino - who remain without a victory since September 22 - in the standings.
Genoa picked up their fourth win out of six games since Gian Piero Gasperini took over as coach with a 2-0 success over Hellas Verona.
The Rossoblu's third win in a row, secured by goals from Daniele Portanova and Juraj Kucka in the first half, lifted them to within five points of their opponents and into a much healthier position than the one they found themselves in when Gasperini took charge.
Hellas tried to find a route back into the game in the second half but they found Genoa goalkeeper Mattia Perin in fine form to keep them out.
On Saturday night, Inter Milan claimed a 2-0 win over Livorno which saw Javier Zanetti make his return to action at the age of 40.
An own goal by Inter Milan's own goalkeeper Francesco Bardi, who is playing on loan this season at Livorno, and a late Yuto Nagatomo strike earned the Nerazzurri all three points at the San Siro.
Zanetti entered for the final 10 minutes, earning a standing ovation as he took to the field for the first time since rupturing his Achilles tendon in April.
A first-half Maxi Lopez penalty proved decisive as Catania squeaked a 1-0 win over Udinese.
In a game that saw both sides create a hatful of openings, Lopez's spot-kick after half an hour was enough for Catania to end a five-game winless run stretching back to September.
Defeat for Udinese - who twice hit the post - was their third in four games and keeps them mired in mid-table, while the three points were not enough for Catania to clamber out of the bottom three, although they are now level on nine points with 17th-placed Sampdoria.