Yeovil started brightly but it was Charlton who took the lead against the run of play in the 11th minute when Johnnie Jackson fired home Lee Martin's cross.
The Glovers were deservedly back on level terms six minutes later after Sam Williams found Andrew Williams and his drive from 20 yards flew past Addicks keeper Rob Elliot.
Charlton were back in front after 26 minutes when Matt Fry played in Therry Racon inside the area and the midfielder's low shot gave John Sullivan no chance.
Yeovil equalised for the second time nine minutes into the second half when a free-kick was swung into a packed penalty area and the unfortunate Gary Doherty fired past his own keeper.
Charlton were reduced to 10 men in the 71st minute after skipper Christian Dailly saw red for an elbow on Sam Williams, but the visitors also went a man down 14 minutes later when Paul Huntington fouled Akpo Sodje in the area and Jackson stepped up to score the penalty.