Fuster netted the only goal of the game in the 72nd minute at the Luzhniki Stadium in Moscow but veteran goalkeeper Carroll, on his debut, also played a key part in the win by saving a penalty from Bebras Natcho with his first touch after coming on in the 73rd minute.
The 34-year-old Carroll, who only joined the club last month, was called into action after first-choice goalkeeper Balazs Megyeri had been shown a red card in conceding the spot-kick with a foul on Gokdeniz Karadeniz.
The 10 men of Olympiacos then held out for the final 17 minutes at the Luzhniki Stadion to record a victory which put them in pole position to progress to the next phase.
Rubin, playing their first competitive match since last year due to the winter break in Russia, found themselves on the back foot early on as Olympiacos enjoyed the better start, but Fuster and Ivan Marcano were unable to make the most of their openings for the visitors as the first half finished goalless.
The second period provided more incidents, beginning with a Nelson Haedo Valdez effort for Rubin in the 47th minute that forced a good save out of Megyeri.
Valdez then saw a long-range effort go harmlessly wide before Olympiacos took the lead midway through the half.
Rubin did not clear Giannis Maniatis' inswinging corner and Spaniard Fuster took advantage to strike a rising shot into the net, with Natcho unable to keep the effort out on the line.
That away goal was just what Olympiacos would have been hoping for, but things could have turned around dramatically 60 seconds later when Rubin won a penalty and the visitors were reduced to 10 men.
However, former Manchester United and Northern Ireland goalkeeper Carroll climbed off the bench - with goalscorer Fuster the man to make way - to superbly keep out Natcho's spot-kick and keep his side ahead.
Rubin coach Kurban Berdyev sent on striker Obafemi Martins and midfielder Alan Kasaev in the closing stages in a bid to grab a late equaliser but they could find no way through as Olympiacos held on for the victory.
