Former Burton forward McGurk nodded in Jayden Stockley’s perfect cross before keeper Lee Grant fumbled over his own line.
Jason Shackell headed an equaliser for the Rams, but two minutes later teenager Chaplin had fired Pompey into the second round.
Portsmouth did not looked overawed by their Sky Bet Championship opponents as they dominated the first half.
Kyle Bennett, a summer signing from Doncaster, looked the liveliest in the opening period and saw two low chances drift narrowly wide.
The visitors pushed towards the end of the half as Andreas Weimann turned and volleyed wide before Johnny Russell had a low cross cleared off the line by Matt Clarke.
Weimann struck the side netting at the beginning of the second half for the away side before Pompey took a deserved lead through McGurk.
The hosts nearly doubled their lead through Stockley and Ben Davies’ quick-fire low efforts.
Shackell scored the first Derby goal of his second spell at the club in the 74th minute with a climbing header, but Chaplin’s second senior goal from an angle soon upset the visitors.