Ward will replace Mark McGhee, who was sacked after Saturday's 4-1 loss at York, and will return to the club initially until the end of the season.
Chairman Nick Higgs told the club's official website: "Initially, John's contract will be until the end of the current campaign, but it is the intention to extend his contract, with the agreement of both parties at the end of the season.
"It is vitally important that we bring somebody to manage the club that has Rovers connections and understands the Bristol Rovers values and John is the perfect candidate for this role.
"Hopefully, under John Ward's guidance, we will return the pride to our club which has clearly been lacking on the pitch for the past few years, and to this end, I ask the players, staff and fans to get behind John and help him take us forward."
Ward will return to Rovers having previously managed the side between 1993 and 1996, and throughout his three-year spell in charge he won 64 out of 149 league matches.
Ward, who began his managerial career at York City in 1991 after a playing career that saw him turn out for Lincoln, Watford and Grimsby, will become Rovers' fourth permanent manager in just two years.
Thrilled about his new appointment Ward said: "The excitement was the thing that convinced me that this was the right thing to do.
"With my hand on my heart, Plymouth was a great job and a nice job. But this one puts my neck on the block. Part of me thinks I'm crazy, but part of me is excited about this fantastic challenge.
"I couldn't turn Bristol Rovers down. Had it been any other football club then I'd have probably said no and that I had a commitment to Plymouth Argyle."
Don't miss any live and exclusive football coverage this festive season. Click here to visit Sky Sports' Festive Calendar