Boss Neal Ardley has made his first capture of the January transfer window by landing 28-year-old Sweeney following his release by the League One club.
Sweeney, who started his career with Millwall and played in the 2004 FA Cup final against Manchester United, has also had spells with Leeds, Grimsby and Stoke.
Ardley told the Dons' website: "Peter has played at a high level and played as a youngster for Millwall against Man United.
"He has also played for other clubs at a higher level and he is a fantastic signing for us. He wanted to come back to London for family reasons and we are pleased to give him that opportunity.
"We were looking at signing him in November on loan but we had a lot of players on loan then. We have kept tabs on him since then and Bury manager Kevin Blackwell has been a big help in making this happen. He is an experienced central midfielder with a wonderful left foot."
