The Iceland international first moved to the Liberty Stadium on a six-month loan from German club Hoffenheim in January 2012.
He went on to score seven goals in 19 appearances as the Swans established themselves back in the top flight.
The Welsh club were understandably keen to secure a permanent deal for the 23-year-old and had opened discussions with Hoffenheim during the close season.
However, those talks collapsed once boss Rodgers opted to leave the club and join Liverpool at the start of June.
The Swans since have gone from strength-to-strength under new manager Michael Laudrup, but Sigurdsson says Rodgers was always the key to any deal.
"I wasn't close to joining Swansea permanently because Brendan obviously left and that changed everything," he said.
"I didn't know who would be coming in and what would happen but what did happen was obviously a good thing for the club because they won a cup this season and are doing well in the league. The club has pushed on from last season."
Sigurdsson did end up making a permanent summer move to the Premier League, sealing a transfer to Tottenham for a fee in the region of £8million.
The midfielder has not enjoyed the same success in a Spurs shirt so far this campaign, scoring just once in 26 league games.
He blames a slight positional change for his lack of goals so far this year, but insists he is happy to play anywhere.
He added: "It's different from last season when I was playing more central behind the strikers but if he [boss Andre Villas-Boas] wants me to play left or right then I'm more than happy to do it. As long as I'm playing I'm happy."