Everton were ahead after five minutes thanks to Aroune Kone’s near-post finish, but there was controversy just before Newcastle’s equaliser at St James’ Park.
Cisse reacted to tussling inside the area by elbowing Seamus Coleman in the face, but with the referee Craig Pawson failing to see the incident, the striker converted from eight yards moments later to bring the hosts level.
Newcastle were ahead shortly after the break through Perez, who slotted into the bottom corner from the edge of the area, before substitute Ross Barkley’s mistake handed Colback the ball on a plate for a two-goal lead.
Kevin Mirallas halved the deficit with six minutes remaining, but Newcastle held on for three points after a nervy finish.
Everton have now lost their last three games in the Premier League, leaving them in 12th.
But they started the brighter on Sunday, and went in front thanks to Kone, who started at the expense of Romelu Lukaku.
Aiden McGeady released Seamus Coleman down the right, and the Irishman’s pin-point cross was converted high into the net by Kone, who had found space ahead of two Newcastle defenders.
Newcastle reacted well, but Moussa Sissoko’s heavy touch let him down after being put through on goal by Cisse.
Just after the half-hour mark, Cisse was involved in a tussle inside the penalty area with Coleman, elbowing the Everton man in the face having been held from a corner. Referee Pawson missed the incident, and it proved costly just a minute later.
Cheick Tiote’s cross deep into the area was turned back at the far post by Mike Williamson for Cisse to convert, wrapping his foot around the ball eight yards out.
The hosts could have been ahead in first-half injury time, but Daryl Janmaat’s effort six yards from goal at the far post was blocked well by Sylvain Distin.
Newcastle nearly scored straight after the restart, Joel Robles doing well to turn the ball away after his own defender Antolin Alcaraz had knocked the ball goalwards.
But the hosts got their lead on 51 minutes after Tiote found Perez on the edge of the area, turning before cooly slotting low into the left corner despite being in a confined space.
Samuel Eto’o wasted a good opportunity for Everton after being found by Leighton Baines 15 yards from goal, and the visitors were made to pay again.
Substitute Ross Barkley failed to control a through ball into the box from Janmaat, gifting it to Colback for the midfielder to push through the legs of Joel Robles for his first Newcastle goal.
Kevin Mirallas, another replacement, made it a nervy final six minutes for the hosts, chipping over Alnwick with style after a fine through ball from Baines.
And as both sides attacked in the late stages of the match, it was Newcastle who could have made matters safe, Perez hitting the post after a superb driven effort from the edge of the area.
Newcastle: Alnwick (6); Janmaat (7), Coloccini (6), Williamson (6), Dummett (7); Perez (8), Sissoko (7), Tiote (6), Colback (7), Gouffran (6); Cisse (6)
Subs: Cabella (6), Riviere (6)
Everton: Joel (6), Coleman (7), Distin (6), Alcaraz (5), Garbutt (6), Baines (7), Barry (6), McCarthy (5), McGeady (6), Eto’o (5), Kone (6)
Subs: Lukaku (6), Mirallas (7), Barkley (5)
Man of the match: Ayoze Perez
Watch extended highlights of Newcastle v Everton on Sky Go.