Radoslaw Majewski's first-half goal secured the win for Forest and ended their seven-game unbeaten league run, but Wednesday had two goalscoring opportunities waved away when Danny Collins and Jamaal Lascelles both appeared to handle the ball in the second half.
Oliver did not award either penalty and Jones was left reeling at the decision.
"I'm confused because I was talking to the assessor before the game," he said. "One, I dont know why we have a Premier League referee and secondly the assessor told me that if the ball changes direction then it's a penalty.
"He gave a handball when it hits a guy's arm from a yard away, then their lad lifts his arm and the ball moves and he didn't give it. That's the little bit of a break that we wanted. But the referee has to spot one of them."
Commenting on his side's Hillsborough performance he said: "I think in the second half we gave it a go, in the first half we gave the ball away too much.
"In the first half our passing wasn't crisp enough, the final 25 minutes was one-way traffic but we just lacked the quality. We were lucky to come in 1-0 down at half-time to be honest.
"That's two defeats in 13. If you'd have said that beforehand we'd have taken it, but we're just disappointed because if we'd have won today we could have moved up four or five places. We felt it was a missed opportunity."