Eunan O'Kane scored the game's only goal when he tapped in the rebound after a Ryan Fraser shot had been pushed out by Pompey goalkeeper Phil Smith.
Whittingham said: "I was pleased with the way the team played for the whole 90 minutes.
"This is a tough place to visit and the team gave a decent account of themselves, although we are clearly disappointed to go out in the first round.
"We made six changes to the team and it will take a while for the team to gel together. I want to give every player a chance."