The Bluebirds looked set to hold on for a point after Juan Cala's equaliser cancelled out Gerard Deulofeu's opening goal.
Seamus Coleman then netted an injury-time winner with a miskicked shot, but Solskjaer was still proud of the character shown by his players.
And Solskjaer was unhappy that referee Roger East had waved away appeals for a spot-kick late in the second half when Wilfried Zaha's mazy run was brought to an end by a Sylvain Distin challenge.
"It was a fantastic effort in the second half. We were brave, got on the ball, created chances and should have had a penalty," the Cardiff boss told Sky Sports.
"It was a certainty. I don't know what we have to do to get a penalty in this league. He goes past two or three and Distin trips him up."
Asked about his side's performance, Solskjaer added: "It was fantastic. We put our bodies on the line and then they get a goal at the end. He mis-hits it and it goes in.
"David Marshall was absolutely fantastic and the two centre-backs, so I couldn't ask for more."
The result leaves Cardiff three points from safety and they must take on title hopefuls Liverpool next weekend, but Solskjaer still believes they can beat the drop.
He said: "The attitude is different class at the moment and we know we can go to places like this and get points, which we should have had.
"Every game is a huge game but if we have this attitude until the end of the season we'll be fine."