Dervite's deal at The Valley ends on June 30 and he will then link up with Wanderers on a three-year contract ahead of pre-season.
The 25-year-old centre-half becomes manager Dougie Freedman's fifth new arrival of the summer, following moves for Neil Danns, Liam Trotter, Liam Feeney and Andy Robinson.
Dervite, who played alongside Freedman while on loan at Southend in 2009, told the club's official website: "I have played in the Championship for the past two years now, and during my first season it was important for me to adapt to the league.
"In my second year I felt more comfortable and I think that people could see that in my play.
"Now I'm with Bolton, I hope that I can carry on improving and developing as a player."
Dervite made 75 appearances for Charlton, scoring five goals.
He began his career with Tottenham before leaving to sign for Spanish side Villarreal and has also represented France at every youth level through to the Under-21s.