Carroll, who joined Liverpool in a £35million deal from Newcastle in January 2011, missed the Hammers' 4-1 defeat at Anfield in December but has scored two goals in his last four matches since returning to fitness.
Sunday's Sky Live game gives him the chance to impress former boss Brendan Rodgers, who took a £20m hit on the club's record signing when off-loading him last season.
Despite the obvious focus on Carroll, Agger has warned his team-mates to expect threats from other areas of the park as West Ham look to cement the Premier League status for another season.
"West Ham is a tough place to go. They have a few more than Andy, it is a good strong team," he said. "You cannot underestimate anyone, we see that week in, week out.
"I think their whole team will give us a hard time, but we have to focus on our game and the way we play. The most important thing is to win the next one."
Liverpool took over top spot last week with a 4-0 victory over Tottenham - their eighth successive league win - and Agger admits the club is looking no further than the 'next game'.
"The most important thing is to be top of the league at the end of the season as the league is so tight," said the Dane.
"I think we showed confidence against Tottenham, but the most important thing is to stay focused and know your job.
"We wanted to win our last game and that was what was in our head and now we have prepared for the next one."
Watch West Ham take on Liverpool live from 3.30pm on Sky Sports 1 this Sunday