The Premier League giants were the first clubs out in the draw and the match could mark Liverpool forward Luis Suarez's return to action after a 10-match ban.
The 26-year-old was handed a 10-match suspension after biting Chelsea defender Branislav Ivanovic last season.
Eager to leave Liverpool this summer at one point, it now looks like he may be staying, and his return would obviously add extra interest to an already intriguing tie.
League One Swindon knocked out Championship QPR on Tuesday night and the Robins have been rewarded with a glamorous home tie against Chelsea, while Bristol City, who saw off Crystal Palace, will be away at Southampton.
Swansea lifted the Cup last season and they have been handed a potentially tricky tie at Championship side Birmingham.
Aston Villa will be at home to Spurs while other all-Premier League ties are West Ham against Cardiff, West Brom v Arsenal and Fulham-Everton.
Manchester City have been drawn at home to Wigan and Newcastle's home tie with Leeds could be an interesting encounter.
Capital One Cup third round draw:
Man Utd v Liverpool
Sunderland v Peterborough
West Ham v Cardiff
Man City v Wigan
Burnley v Nottingham Forest
Newcastle v Leeds
Southampton v Bristol City
West Brom v Arsenal
Swindon v Chelsea
Tranmere v Stoke
Watford v Norwich
Aston Villa v Tottenham
Hull v Huddersfield
Leicester v Derby
Birmingham v Swansea
Fulham v Everton
Ties to be played week commencing September 23.