Long came back to haunt his old club as he won a penalty and scored a goal in Hull City's 2-0 win over West Brom at the KC Stadium on Saturday.
West Brom have struggled since the departure of Long and find themselves three points above the drop zone after a terrible slump in form since the turn of the year.
Foster admits he is perplexed as to why the club would decide to sell one of their top performers in January and admits his departure has been a big blow for everyone at The Hawthorns.
"Sometimes I think why did we let him go to be honest with you," said Foster. "It's not even hindsight.
"He was one of our best players from the last couple of years and it's disappointing that he's no longer here. When he went, it was obviously a big blow for us because we know what Shane does.
"He will give you 110% every single minute of every game. He's a fantastic player and we would love to have him now."
"If you had asked the players, we would have liked to have seen him stay.
"It's a business at the end of the day, so it's out of our hands."
Foster also defended Long after he was accused by West Brom fans of diving to win Hull's penalty, from which Liam Rosenior scored after Nikica Jelavic's spot-kick was saved by Foster.
Long went down in the box under a challenge from Craig Dawson even though it looked there was minimal contact between the players, but Foster insists the Republic of Ireland international is no cheat.
"I don't think it was a penalty and I don't think Shane was looking for a penalty," added Foster.
"He's a great guy. I know the fans weren't too happy with him, but Shane's not like that. He's not one of those guys who will dive.
"He's a real, honest hard-working guy. He knows what his role is and he does it week in, week out."