The Millers cancelled the 26-year-old's contract by mutual consent on Monday and he joined the Robins just hours later.
"We are delighted to have signed Jason - this was a no brainer," manager Mark Yates told the club's official website.
"We were fairly surprised when he was put our way but we are very happy to have signed him on a two-and-a-half-year contract.
"He is a leader, a winner and he will add something extra to our already talented midfield."
Taylor began his career at Oldham before a successful spell with Stockport.
He left Rotherham after four years and over 120 appearances and will be available for Cheltenham's clash with Torquay this weekend.