The Lions, who had taken three points from their opening three games of the campaign, hit their stride against Middlesbrough from the off and raced into a two-goal lead after 35 minutes.
Skipper Liam Trotter got the ball rolling with a precise 20-yard effort in the 25th minute and Keogh got his first of the afternoon and the season 10 minutes later when he reacted quickest to his shot thundering into the post.
Boro, who started the day in the top six, gave themselves hope before half-time through Marvin Emnes' second of the season, but Keogh's second from close range in the final five minutes made sure of the points for the Lions.
Boro will have not appreciated the lunchtime kick-off in south-east London, having not arrived home from their midweek Capital One Cup victory at Gillingham until 4am on Wednesday.
And after another lengthy journey back down to the capital their weariness began to show as the Lions pressed and Keogh volleyed straight at Jason Steele before missing an even better chance when denied by the Boro keeper from close range.
But the home domination eventually paid and Trotter gave his side the lead in the 25th minute when he latched onto a loose ball and fired into the top corner from 20 yards.
That became 2-0 10 minutes later as Keogh, who had been a constant threat throughout, was found in the penalty area and lashed a shot that beat Steele, but thundered into the post. However, the former Wolves striker reacted quickest and slotted the rebound home.
Middlesbrough were in danger of being blown away but they were given a lifeline two minutes before the break when Jimmy Abdou fouled George Friend in the penalty area.
Emnes coolly struck home from 12 yards to reduce the deficit.
That goal changed Tony Mowbray's half-time team-talk and Boro came out roaring after the interval, with Merouane Zemmama and Grant Leadbitter having shots blocked.
Boro then had three opportunities to level but Leadbitter fired just wide, Friend's shot was blocked and Emnes cleared the bar from the edge of the area as they searched desperately for a way back into the game.
And they paid the ultimate price with five minutes remaining as Keogh put the game to bed for the hosts when he was on hand to tap home Scott Malone's cross from the left.
Friend immediately tested Maik Taylor in the Millwall goal, but the former Birmingham man parried the danger away as the hosts saw it out with ease and condemned Boro to a miserable journey home.
