The Norwegian striker scored a last-gasp goal at the right end to give Portsmouth all three points on Saturday but he diverted the ball past his own goalkeeper to give Posh a dramatic win - their first away victory of the season.
The visitors led twice following a brace from Lee Frecklington but it seemed as though Benjani had rescued a point for Portsmouth with his first goal since returning to Fratton Park.
Frecklington had given away the penalty which West Ham converted in Posh's 1-0 defeat at Upton Park on Saturday but he broke the deadlock in the fourth minute.
The midfielder hit a snap-shot with the outside of his boot from the edge of the box and Pompey's goalkeeper Jamie Ashdown could only turn the ball onto the underside of the bar before it dropped over the line.
Posh's lead lasted just five minutes before Gabriel Zakuani turned into his own net.
Huseklepp drilled a dangerous angled shot across the box which was heading wide until Zakuani volleyed it past Posh's goalkeeper Paul Jones with an attempted clearance.
Yet Darren Ferguson's men, who had not scored on the travels in over a month, were ahead once again after 21 minutes.
The hosts failed to clear their lines and the ball eventually fell to Frecklington who promptly lashed the ball into the top corner for his second of the evening.
Posh seemed intent on extending their lead rather than being pegged back again, with Grant McCann and Paul Taylor both going close.
Another Peterborough goal always looked likely and Ashdown failed to keep hold of a thunderous George Boyd strike, but Emile Sinclair could not profit from the rebound.
There was still time in the first half for Ashdown to induce more panic into the Fratton Park faithful when he gifted Posh possession with a woeful pass, but the visitors could not capitalise.
Pompey started the second-half sharper and Zakuani redeemed himself following his earlier error, throwing himself in-front of Benjani's goal-bound effort.
But the Zimbabwean went one better in the 54th minute to pull the hosts level again. Huseklepp was again the architect with another low ball across the box which Benjani prodded home for his first Portsmouth goal since 2008.
The momentum was now with the home side and Jones had to deny both Hayden Mullins and David Norris before Zakuani and Craig Alcock cleared the ball off the line following a flurry of Pompey chances.
But Peterborough snatched a victory at the end when substitute Ryan Tunnicliffe raced clear and Huseklepp slid in to divert his shot past the hapless Ashdown.