Mid-table Colchester beat the league leaders thanks to a determined performance, taking the lead in the fifth minute when Wordsworth picked the ball up 25 yards out and curled a sweet left-footed shot into the top corner.
Charlton pressed for an equaliser and Johnnie Jackson, Bradley Wright-Phillips and Danny Hollands all went close in the first-half.
Moments after the break Yann Kermorgant had a shot deflected wide and Wright-Phillips'shot was parried away by Colchester goalkeeper Ben Williams.
Colchester nearly managed to add a second against the run of play when Gillespie picked up Wordsworth's flick-on, but Charlton goalkeeper Ben Hamer saved well from close range.
But it was Hamer's mistake moments later in the 73rd minute which ultimately condemned Charlton. He dallied on a clearance allowing Gillespie to block the ball and it ricocheted into the empty net.
