The club's record signing, who has been out of action this season due to a foot injury, made his comeback against Cardiff and then appeared as a substitute again in Saturday's 3-1 home defeat by Newcastle.
The loss dropped the Hammers back into the relegation zone but boss Allardyce is refusing to panic by rushing Carroll, a £15m buy from Liverpool, back into the side before he is 100 per cent ready.
However, he does concede the England striker will be given another run-out against Manchester City in Tuesday's Capital One Cup semi-final second leg, a tie where the Londoners trail 6-0 from the first game at the Etihad last week.
"Andy is going to compete in a game for the first time just a few days after playing on Saturday," he said.
"This is something we are looking forward to in the hope that there are no problems and knowing he is getting even closer to a full 90 minutes.
"Our other scenario, from a defensive point of view, is to continue to try and get Joey O'Brien, Winston Reid and George McCartney back playing as quickly as we can.
"And we also have Ricardo Vaz Te to come back. He has been out for around 12 weeks and people tend to forget about the long-term injury list to people who are very important to our team, and squad.
"So all the players are looking close to returning from injury which is very important for the next 16 Premier League games to come after this."
And Allardyce admits trying to mix the day-to-day running of the club while trying to stengthen his team in the transfer window is proving almost impossible.
"There are so many balls in the air at this time of year and in the meantime you have to put your team out on to the field playing as well," he said.
"That is the difficult part from my point of view. I think the people outside of here think the pressure on David (Moyes) is greater than on me but from my point of view I believe it's equal.
"It's not just about now. This club wants to move into the Olympic Stadium in two years so we have to be looking to build for that.
"But, hopefully, come the end of the season we can look back and say 'well that was a tough season but we are wherre we need to be... the Barclays Premier League'."
Watch West Ham take on Manchester City at Upton Park live on Sky Sports 1 from 7.30pm on Tuesday