Welsh striker Williams, 21, only arrived at Colchester on a month's loan from Aston Villa on Monday, but after 17 minutes of Tuesday's quarter-final tie, he showed some sharp finishing with a first-time shot at the end of a pass from Kem Izzet.
Bournemouth's stand-in goalkeeper Ryan Pryce made an outstanding save to deny Williams earlier and Izzet missed an open goal from three yards after being picked out by a pass from Scott Vernon.
The Cherries improved in the second half with the introduction of former England winger Darren Anderton as a substitute in place of Mark Molesley.
Brett Pitman went closest to equalising in the 62nd minute when a left-foot shot was turned round the post at full stretch by goalkeeper Mark Cousins.
Colchester almost scored another with seven minutes remaining when a free-kick from 25 yards by Mark Yeates hit the outside of a post.