Britton made the warning after Saturday's Barclays Premier League defeat to West Brom.
The Swans head into their last nine games of the season just four points above the drop zone with a number of testing fixtures to come.
Britton told his club's official website: "Losing on Saturday makes it harder for us as it was to a direct rival.
"It means we are very much part of the relegation battle and it's up to us to get out of it.
"Over the last few years we have had nothing but success at this football club, but now we have a different challenge on our hands."