The quartet, all Academy graduates, have been deemed surplus to requirements by boss Steven Pressley having failed to make an impact this season, with only centre-back Henderson managing two first-team appearances, both of which came in September before the Scot's arrival.
Pressley said: "I feel that Shaun, Will, Josh and Joe are at a point in their career where they should be really pushing for first-team football.
"It's a difficult decision to let any player go but we will do all we can to help them in their future careers.
"From the club's point of view, it's massively important to me that we continue to develop our own and ensure that our younger players are staking a claim for a place in the first team on a regular basis."
The quartet join Richard Wood, Jordan Stewart, Stephen Elliott, Nathan Cameron and Danny Philliskirk, each of whom were released at the end of April.