The Blades began the day six points adrift of safety with two games to play, but they looked as if they would take their fate to the final day as they led for much of the second half and other results were going their way.
But Danny Haynes' late goal, his second of the game, condemned United to the drop and next season they will be playing third tier football for the first time since 1989.
Haynes' cool finish had put the Tykes ahead in the 15th minute but United rallied and goals from Matt Lowton and Lee Williamson raised hopes of pulling off the great escape.
The late goal by Haynes did not matter anyway as Doncaster and Crystal Palace both hit back to pick up points, which meant United would have been down even if they won.
For the Barnsley fans, who have had to endure a flat end to the campaign, sending their bitter South Yorkshire rivals down was a moment to savour, but their side have won just one of their past nine games and will be looking forward to the close season.
Despite winning their previous two games there was an air of inevitability around Bramall Lane, with United knowing that even another victory may not be enough to keep them up.
It was a pulsating start to the game and the hosts twice could have taken the lead in the opening five minutes, with Williamson's low drive saved by Luke Steele and Stephen Quinn firing an effort inches wide of the angle.
Barnsley hit back, though, and wasted a golden chance when Jacob Mellis blazed over after ghosting through the Blades defence.
But the Tykes made amends and went ahead with 15 minutes on the clock when Haynes raced clear following Harry Maguire's slip and coolly slotted past Steve Simonsen for his fifth league of the season.
Neither defence were covering themselves in much glory with several errors leading to scrambles and last-ditch tackles in both penalty areas.
So it was no surprise when the Blades drew level six minutes before the break when Lowton powered in a header from Williamson's wicked free-kick.
United were building up a head of steam and Williamson had a goalbound effort blocked by Nathan Doyle before Lowton spurned a superb chance to double his tally after shooting well over after a mazy run as Micky Adams' men could not find a breakthrough before the break.
The home side continued in the ascendancy after the restart and they took the lead thanks to a calamitous error by Barnsley goalkeeper Steele.
A loose ball fell to Williamson on the hour and he drilled a low effort that Steele inexplicably let through his hands and into the net.
United could not capitalise though and after Simonsen had pulled off a sublime save to deny Kieran Trippier, Barnsley drew level with six minutes remaining.
Haynes stole in to head home Matt Hill's cross and ensure the curtain came down on Championship football at Bramall Lane in disappointing fashion for the hosts.