Lukaku has impressed while on loan at the Baggies this campaign, netting 13 goals for Steve Clarke's side.
The Belgian international is thought to be willing to go out on loan again next season, but to a team playing in Europe so that he can continue his development.
Olsson admits it is unrealistic to think they can keep Lukaku at The Hawthorns and concedes he will be hard to replace next term.
"Romelu has been massive for us. It's not realistic to think we keep him," said Olsson.
"Someone who scores 13 goals for a club like West Brom, with all respect, it's a great achievement. He'll be difficult to replace but that's the challenge."