Taylor struck a superb shot past U's goalkeeper Sam Walker in the sixth minute of time added on to snatch the Saddlers a point.
Walsall started well, with both James Baxendale and Craig Westcarr going close early on.
But Colchester took a seventh-minute lead when Freddie Sears struck with a fine strike from the edge of the area after being found by Gavin Massey.
Walsall went close midway through the half when James Chambers' long-range effort clattered the woodwork before Andy Butler's header was cleared off the line by Matt Taylor.
Alex Gilbey fired wide for Colchester, but Walsall came agonisingly close to equalising on the stroke of half-time when Sam Mantom's fine effort hit a post.
The second half was an even affair, with much of it played out in midfield and little in the way of goalmouth opportunities for either side before Taylor's superb strike in the dying moments secured a point for the Saddlers.