The Sweden left-back, who can also play in midfield, returns to the Premier League after Rovers accepted an undisclosed offer.
Manager Chris Hughton told the club website: "We're delighted with the addition of Martin. He's been playing in England for seven years and is well acclimatised to the Premier League.
"He can also play anywhere on the left hand side so he brings us good flexibility in that sense, plus at 25 he's at an age where he has some experience and also some time to improve.
"At this time it's about improving the squad that we have and Martin certainly does that."
Olsson, who arrived at Ewood Park as a 17-year-old eight years ago, leaves with a record of six goals in 143 appearances.