A goal in each half from Brennan Dickenson and Aaron Morris secured a 2-1 win for the Gills, before Glovers' right-back Seth Nana Ofori-Twumasi was sent off.
Taylor said: "There were a lot of positives and there was some good football being played at times.
"The players were more calm on the ball and, okay the first goal was a bit lucky, but we played some decent football.
"Both teams got beaten on Saturday and were desperate to get something.
"I thought our shape looked good and when we played it simple we did it well.
"We played 3-5-2 and that is something I have been wanting to try for a while and I thought tonight was a good opportunity. I played it with the England Under-21s between 1996-99 so it is one that I know well and a lot of people like it.
"It is a good system when you have got the ball and we have got to make sure we do that."
Yeovil were undone by a defensive calamity after 23 minutes when goalkeeper Artur Krysiak and centre-half Aaron Martin crossed their wires to allow Dickenson in to slide home the opener.
The equaliser came in the 56th minute when Simon Gillett gave keeper Glenn Morris no chance with a thunderous strike from a well-worked free-kick.
But the Glovers failed to hang on and were behind again when Aaron Morris was gifted the time to fire a perfect strike past Krysiak just before the hour mark.
Yeovil were reduced to 10 men after 62 minutes when referee Darren Sheldrake showed a straight red card to Seth Nana Ofori-Twumasi, who was adjudged to have left a late tackle on Luke Norris.
Home manager Gary Johnson said: "We were disappointed with the couple of mistakes for the goals. We got ourselves back into it but the we have a mountain to climb with the sending off.
"I have got five or six willing to go beyond the call of duty and others that need to catch up and have not quite shown the desire and passion to win football matches. That is something I have got to look at, but the season is not over."
On the red card the Londoner said: ""I will have a look at the video of the challenge but I can tell you whatever I see it is not going to be malicious because he doesn't do that.
"I am disappointed for him because he has worked hard to get his spot and I thought he was doing well.
"When someone stays on the floor until a card is shown and then gets up and plays then you are disappointed."