Millwall took the lead in the 52nd minute when Worner gathered a low return pass but released the ball fearing his momentum would take him outside the box, allowing Keogh to slot home into an empty net.
Steve Lomas made seven changes to the side that lost at the weekend and it was one of those in Jermaine Easter that should have broken the deadlock in the first half.
On 28 minutes the striker picked the ball up on the edge of Wimbledon's box and unleashed a curling effort heading for the top corner but Worner was equal to it.
Easter went close again on 34 minutes when Nicky Bailey, making his full debut, found him with his back to goal in the area but his low effort was easily blocked.
Early in the second half James Henry fired over from six yards out and after gifting Keogh the opener, Worner denied Woolford with a smart save with his legs.
In the 77th minute Woolford fired home a deflected Easter cross as Millwall were cruising through, but substitute Luke Moore had other ideas and nipped in at the near post in in the 89th minute to slot past debutant Stephen Bywater.
But Bywater was then equal to a stoppage-time Jack Midson effort to ensure Millwall's path into round two.