The Hammers paid Liverpool a club record £15million for Carroll in the summer but he has not played since.
He suffered a double fracture of the right foot whilst he was recovering from a heel injury, and in his absence West Ham have struggled for goals, having scored just 13 in 15 Premier League games this season.
"Had we known he would be out for this long, we would not have signed him," Sullivan told Football Focus.
"We are not a rich enough club to deal with that. You know any player can get injured, but we can't buy a player knowing he is going to be out for half the season.
"When we signed him we were assured by the medical staff that the very, very latest he would be back was 1 September. That would have meant he would only miss two league games."
Carroll is three weeks away from being fit to play and Sullivan is still hopeful that the England international can fire the Hammers to safety.
"If you ask me the same question at the end of the season, I hope I will be able give you a different answer and say 'Yes, it was the right decision to sign Andy Carroll'," he said.
"He is a fantastic player and on his day he is unplayable. We love him and that is why we signed him. He is desperate to come back; he is sweating blood in training."
Until the goals begin to flow, Sullivan admits West Ham are in a dogfight at the wrong end of the table.
"I am deeply concerned," he said. "Anybody who looks at where we are in the table and is not concerned are kidding themselves."
However, he is confident Hammers boss Sam Allardyce is the right man to lead them out of trouble.
"I really believe Sam is the man for the job," he said. "If he wasn't our manager he is exactly the type of manager you would be bringing in to be the manager. Where do you get someone better than Sam to do the job that is now necessary?
"If you asked me what kind of season Sam has had up to now, I would say 'disappointing'. But let's judge him at the end of the season.
"I think the Newcastle manager would have been sacked by most fans after a month of the season but now he is flavour of the month."