Solskjaer, who was appointed Cardiff boss on Thursday, carved a reputation as a super-sub during his career as a Manchester United striker and his changes helped to inspire his team to victory.
Papiss Cisse had marked his first appearance in the FA Cup by giving Newcastle a 62nd minute lead when he squeezed the ball home from a narrow angle after Moussa Sissoko's shot had been blocked.
Craig Noone was brought on for Peter Odemwingie and, within moments of his introduction, the winger drew Cardiff level on 73 minutes by firing a left-footed effort from 25 yards into the top corner.
With 10 minutes to go, fellow substitute, and former Sunderland striker, Fraizer Campbell headed home Peter Whittingham's corner at the far post to send Cardiff into the fourth round.
Newcastle boss Alan Pardew indicated he would make changes for a fifth game inside 15 days, and he was as good as his word as seven of the men who started the 1-0 league defeat at West Brom on New Year's day made way.
However, he was still able to name a strong side which included Rob Elliot, Steven Taylor, Mapou Yanga-Mbiwa, Vurnon Anita, Cisse and Hatem Ben Arfa.
Solskjaer, who is still getting to know the players he inherited from Malky Mackay, opted for five changes and, after a low-key start, it was the Bluebirds who settled into a rhythm the quicker.
Indeed, they thought they had taken a ninth-minute lead when skipper Mark Hudson powered a header past Elliot from Whittingham's corner, only to be penalised for a foul.
With Whittingham and Kim Bo-kyung pulling the strings in central midfield, the visitors mounted a sustained period of pressure during which Kim and Don Cowie, who had earlier headed weakly at the keeper, both fired over.
Anita belatedly tested David Marshall at the other end with a tame 17th-minute shot from distance, but it was Ben Arfa who took up the baton as the half progressed.
The France international sent a 19th-minute effort just wide and then rattled a post with a thundering shot after twisting away from Whittingham and Odemwingie six minutes later.
Given their manager's complaints over the recent fixture schedule, Newcastle returned determined not to add a replay to their list of commitments, and Ben Arfa took it upon himself to ensure that was the case.
The Frenchman picked up possession after Cisse had done well to collect Davide Santon's ball over the top and powered his way into the box and shot, only to see Marshall somehow turn it on to the top of the crossbar.
Marshall had to claw Sissoko's looping header from under the bar a minute later with the Magpies going for the kill.
The pressure finally told with 62 minutes gone when Yoan Gouffran played the ball over the top for Sissoko, who delayed and saw his effort blocked by the sliding Ben Turner.
However, Cisse followed up and, despite Hudson's best efforts, turned the ball past Marshall for his third goal of the season.
Cardiff might have been back in it within five minutes but Cowie flashed a header wide of the post from Gunnarsson's driven cross, and they were unfortunate not to be level with 20 minutes remaining when Campbell struck the foot of the post and Odemwingie skied the rebound.
But the visitors were level within three minutes when Noone, who had only been on the pitch for seconds, smashed a left-footed piledriver past Elliot's despairing dive.
And there was worse to come for Newcastle when they left Campbell criminally unmarked to power home an 80th-minute header from Whittingham's corner to win it.
Solskjaer will now get down to work in earnest with Premier League survival his main focus, while opposite number Pardew will reflect upon another early exit from a competition in which he has won just one game in his three years on Tyneside.