Taylor’s 73rd-minute leveller only came after strike partner Adam Le Fondre’s earlier penalty had been spectacularly saved by home keeper Danny Potter.
While in the 88th minute Taylor looked to have won it but could only watch as his shot hit the inside of a post and ran along the goalline before being cleared by central defender Guy Branston.
Eunan O’Kane had put Torquay ahead in the 17th minute after Tom Newey had headed Chris Zebroski’s goalbound shot off the line.
Millers number one Andy Warrington made a spectacular diving save to push Billy Kee’s thumping drive out of the top corner as the home side went in search of a second goal.
That stirred Rotherham into counter-attack action and it took a brilliant save from Potter, reaching up to keep out Marcus Marshall’s rising rocket from the left-hand side of the Gulls penalty area. But Potter topped that with a brilliant 59th-minute penalty save to deny Rotherham ace Le Fondre after Marshall was fouled on his way to goal as he broke into the box from the left wing.