Posh had Hogan Ephraim red carded for a foul on Liam Noble just after half-time although they then took the lead with a Lee Tomlin 25-yard free-kick.
David Amoo struck back for Carlisle in the 71st minute, driving home right footed from 12 yards after Tom Lawrence had provided the cross.
A minute later he was on target again, side footing substitute Matty Robson's searching ball from the left into the net from close range.
Peterborough missed a late chance to salvage a point when Mark Little was fouled in the box. Goalkeeper Ben Amos got fingertips to Grant McCann's penalty and the ball rocketed off the right-hand post and back into play.
Ferguson said: "The red card was a bit soft. If they were all like that there could be three or four a game.
"I thought the performance of the officials as a whole was poor. I could give an honest opinion of the referee, but I would get into trouble.
"The standard of officials is really poor, but that's just my opinion.
"At 1-0 I thought we would go on and get another goal and pick them off, but you have got to defend better than that.
"Both goals we lost were poor goals. and nothing to do with the advantage Carlisle had. The second goal was appalling. Then Macca (Grant McCann) has hit the post with the penalty which is unlike him.
"It's my responsibility to pick them up but sometimes players have got to take responsibility and pick themselves up. It's a big challenge for us, but we've gone on runs in the past and we can't shy away from the challenge; life's not that easy."