The Pilgrims are currently 23rd in League Two, just two points clear of bottom club Bristol Rovers, and fired Carl Fletcher on New Year's Day.
The sacking followed a 2-1 defeat at Bristol Rovers which left Plymouth with just one win from 15 matches and they have now turned to Sheridan, who has had success in Leagues One and Two/
The 48-year-old guided Oldham to the League One play-offs and he won the Johnstone's Paint Trophy with Chesterfield as well as leading them to promotion to League One, although they came straight back down last season.
Plymouth owner and chairman James Brent said: "I am delighted to welcome John to Home Park.
"Following a distinguished playing career, John has a proven record of leading clubs to success in Leagues One and Two.
"John impressed the interviewing panel with his commitment to excellence and his approach to moving the club up the league table. He expressed an enthusiasm for attacking football and was confident in his analysis of what we needed to do to succeed.
"John noted his own experience of playing against Argyle at Home Park and the advantage that the Green Army's passion gave to the home side. The Green Army has had a tough few years and deserves better, so let's all get behind John and the team and make Home Park a fortress once more."
Sheridan added: "I am very privileged to get this opportunity to manage the club. Everyone I've talked to says what a good club it is and what potential there is.
"I am very confident I can get us out of the position we are in if everyone - the players and fans - works together. I remember the support from the fans when I played down here with Leeds and Oldham, and that was one of the things that attracted me.
"From my time at Oldham and Chesterfield, I think I know what's required from the players and what we need to do to succeed."