Connell headed home a centre from Warren Cummings after 66 minutes to help Bournemouth knock out a team a division above them - but it was a close affair.
Bournemouth went ahead in the 45th minute with a well-worked corner routine which ended with Brett Pitman lashing in a right-foot shot from the edge of the area.
After Connell's goal Yeovil pulled a goal back in the 86th minute when a pass from Sam Williams was turned in by substitute John Obike. The visitors also hit the post twice through Gavin Tomlin in the 38th minute and George O'Callaghan in the 62nd.
Bournemouth goalkeeper Shwan Jalal had to make a desperate save on his line at the second attempt to keep out a header from Terrell Forbes.
Yeovil goalkeeper Richard Martin denied Connell a goal in the fourth minute with an outstretched foot but made two outstanding stops to keep out a low shot from Lee Bradbury and, in injury time, when Liam Feeney was clear.