Carroll was sidelined with an injury for the first half of last term and did not return to action until January.
He played well at times for West Ham after his comeback but managed just two goals in 16 appearances and Sullivan thinks he can have few complaints about being left out of Roy Hodgson's World Cup squad.
Sullivan reckons it will be a different story this season, though, and has backed Carroll to make a big impact in the Premier League if he can stay fit.
"Andy will come good. He didn't have a pre-season last year," Sullivan told the Daily Mirror.
"He never really got his act together because of that and I think you'll see a much better player next season.
"It was a fair decision to leave Carroll out of the England squad based on what he did last season.
"He didn't do enough when he came back. There were a few games he was devastating in. Against Swansea he set up both of Kevin Nolan's goals. But he knows he didn't do enough to play for England.
"And I'm glad he is not playing for England. I don't want him arriving three weeks late because he has been in Rio all summer. But his time will come."