The 22-year-old signed for the U's in 2010 from non-league side Bury Town and he went on to play 45 times for the club.
After making 20 league appearances in each of his first two seasons, the versatile defender found opportunities hard to come by this season.
He has played just once in the league - back in September at Brentford - while his last outing came in the Johnstone's Paint Trophy defeat to Northampton in October.
A statement on the club's official website read: "Ben had found his first team opportunities limited under boss Joe Dunne and his contract was cancelled by mutual consent on Friday afternoon.
"We wish him all the best with his future football career and thank him for his efforts while at Colchester United."
