Midfielder Jack Cork, who has been linked with a move away from the south coast all summer, climbed off the bench to grab his first Saints goal on his 114th appearance for the club.
Pelle, the summer signing from Feyenoord, also opened his account in stoppage time to settle a match Saints had been making hard work of.
Manager Ronald Koeman saw the second-round encounter as a chance to get his new-look team up to speed and – unlike many Premier League bosses – made just three changes, as well as handing striker Shane Long his full debut.
Beforehand Koeman had ordered Pelle to toughen up as he adapts to life in England – and a chilly Tuesday night down at The Den can certainly be character building so it was no surprise to see him in the starting line-up.
But it was much-changed Millwall, with just goalkeeper David Forde and defender Shaun Williams surviving from Saturday’s Championship defeat by Rotherham, who should have been ahead at half-time.
Byron Webster flashed a header across goal and wide before Magaye Gueye’s composure deserted him in front of goal and he shanked his shot past the post.
Just before the break Jermaine Easter tried an angled volley but Jose Fonte stuck out a leg and deflected the ball over.
The visitors, meanwhile, were easy on the eye but creating next to nothing, with a weak James Ward-Prowse shot which rolled into Forde’s arms the sum total of their efforts in the first half.
It was Millwall who threatened again moments after the restart when Webster fed Easter, who cut inside from the left before rifling a shot narrowly wide.
But that only seemed to wake Saints up and they promptly laid siege on Forde’s goal, with both Dusan Tadic and Pelle forcing the Irishman to tip fierce drives over the top.
And from one of the resulting corners, Southampton broke the deadlock, Cork ghosting in at the near post to nod Tadic’s cross past Forde.
It could have been two moments later when Nathaniel Clyne scampered down the right and crossed for Pelle, but he was denied a first goal in this country when his header shaved the post.
Plucky Millwall pressed for an equaliser and their former Saints midfielder Richard Chaplow went closest when his shot flew narrowly over.
But Pelle settled it in injury time when he raced onto a through-ball and calmly beat Forde from 10 yards.