Jack Baldwin headed Pools in front after 18 minutes, before James Poole doubled their advantage seven minutes after the break.
Pompey youngster Jed Wallace pulled one back for the home side after 67 minutes, but Hartlepool substitute Greg Rutherford netted a third in stoppage time to seal their third victory of the season.
Whittingham handed Shaun Cooper, Adam Reed, James Keene, Dan Butler and Wallace their first League One starts for the club this season and gave Yassin Moutaouakil and Frankie Sutherland their debuts from the substitutes' bench.
He said: "I'd like to look at the shooting stats because I'm sure we had more chances than them, but we didn't put ours away.
"I don't think (goalkeeper) Simon Eastwood had much to do in the second half apart from pick two balls out of the back of the net.
"I'm very disappointed with the result. We had enough chances in the game to score and our defending was poor for the first and second goals.
"We were going for it when they scored the third, but the ones before that I wasn't happy with."
