The Premier League club were reported to have agreed an 8million euros fee for the former Lille man, but the 27-year-old appears keen to remain at the Stade Velodrome.
Payet claims he has always rejected the overtures from the Liberty Stadium side, although he suggested that an offer from ‘a bigger club’ would have turned his head.
The France international told L’Equipe: "With all due respect to Swansea, I want to stay at OM.
"I said no to my agent right after the first contact. I told the president from the start that I want to stay. Despite that, the press still says that I'm hesitating.
"If a bigger club had shown interest in me, I would be gone by now. But I had an average season and you should do more to be coveted.
"In football, you can never say never, but I feel like I'll stay at Marseille, 100 per cent."
Payet, who only moved to Marseille in the summer of 2013, accepts that the club may be prepared to let him go after admitting he did not meet his own expectations last term.
"I'm being pushed out of the club, but I can understand that. There is an economic reality behind that," he added.
"My first season was average. I've expected better from myself and I guess it's also their case."