After three successive wins, the Spireites had given themselves a chance of avoiding the drop and they led 2-1 with 20 minutes left only for Yeovil to hit back.
"I am really disappointed they are going down," said Johnson, whose son Lee had a lengthy spell on loan to the Spireites earlier in the season when he was at Bristol City.
"They are a good club, with some very good players and an excellent manager, so I'm sorry we shall lose them. But that's the way it goes."
Jordan Bowery knocked in his 12th goal of the season to give Chesterfield the lead in the eighth minute, but Yeovil levelled after 26 minutes when Dominic Blizzard was well-placed when Andy Williams pulled the ball back from the byline on the left.
Danny Whitaker's fierce drive had the visiting fans in raptures, but Kieran Agard and Williams were on hand to seal their fate.
