Olsson went tumbling to the ground after being clattered by John Terry in the 11th minute of the Stamford Bridge clash after he was played in by Bradley Johnson.
However, referee Neil Swarbrick waved away his appeals for a spot-kick as the visitors missed out on a chance to score a precious goal in a game which eventually finished goalless.
Olsson told Sky Sports: "I knew I got the touch first and then, as you can see, it is a clear penalty.
"The referee didn't see but I knew I got that touch first and otherwise it would be a different story. What John did right there, it was a clear penalty and it could have given us three points as well.
"We came here to defend and we needed the half chances and the penalty would have saved us as well. But that happens, that's football. We have got one more game and hopefully the luck is on our side."
Chelsea also had a couple of penalty claims waved away including when striker Demba Ba tangled with Norwich centre-back Michael Turner.
"I just saw him running across and I just tried to block him off," said Turner.
"I probably was a little bit too aggressive and he went down and I think I winded him a little bit. But I didn't mean to hurt him; I just meant to block him off really."
The draw left 18th-placed Norwich, who host Arsenal in their final game next Sunday, two points adrift of 17th-placed Sunderland, who have two games remaining.
Turner added: "Obviously we need Sunderland to lose in midweek and then lose at the weekend and we need to win, so that's it.
"We will work hard this week in training with our fingers crossed and hopefully we can pull off a miracle really.
"I think we came here to do exactly what we have done, sit back and wait for the half chances to come along.
"We were hoping for a little bit of luck to maybe put one away, but it just wasn't to be."