Di Canio, whose League One team had already taken the scalps of Championship Brighton and top-flight Stoke in the opening two rounds, were deserved winners against the off-colour Clarets.
"We are happy to go through," the Italian said. "Maybe now we have a chance to go and play at a fantastic stadium.
"I would sign right now to play a Manchester United or Chelsea away. Maybe there is only a one in a million chance (to beat them) but I would like to try."
It was the Clarets who made the brighter start, former Robins favourite Charlie Austin going close to opening the scoring when his powerful early header was cleared off the line by Swindon left-back Jay McEveley.
But on 19 minutes, Scot McEveley turned provider as the home side took the lead against the run of play when his fizzing left-wing cross was neatly half-volleyed home by striker Paul Benson.
Burnley continued to dominate possession - with midfield Chris McCann and Austin both spurning decent opportunities - but it was Swindon who would double their advantage on 41 minutes.
After a succession of misplaced passes, the ball found its way to Andy Williams, who outpaced the static Burnley defence before coolly finishing past Brian Jensen in the Clarets goal.
The second half was more of an even contest with Williams going close for Swindon, while Austin, who was booed loudly throughout the 90 minutes, also wasted further openings for the visitors.
Fifteen minutes from time, Austin finally silenced the County Ground crowd, reacting quickest to a bouncing ball in the Swindon area to tap home from inside the six-yard box.
But Burnley's hopes of a comeback lasted only eight minutes before Troy Archibald-Henville nodded home his first goal for the club to secure a comprehensive victory for Di Canio's men.
The Italian, whose side have now knocked out eight higher-division sides since his appointment, added: "We went through tonight because we deserved it. I don't want to sound arrogant, but we scored three and we could have scored three or four more.
"But today we can smile. We have beaten eight teams above us and that is not an accident.
"We have quality and desire. We have togetherness. And if we keep going, we will have lots of smiles this season - and at the end, maybe a big laugh."
Burnley boss Eddie Howe insists some hard work on the training ground lies ahead for his players after the defeat in Wiltshire.
"It was disappointing because don't get me wrong, we wanted to win tonight. It was a big opportunity to get into the last 16," Howe told the club's official website.
"With two wins in the last three matches you think we have turned the corner and we then seem to fall back again and it's a big challenge for us, because I don't think the ability in the squad is in doubt - but we have to deliver it.
"We had a number of chances tonight and played really well for the first 20 minutes. Charlie [Austin] had a couple of chances and they've had one off the line, but they scored and goals change games and we have rocked a little bit.
"We conceded a poor second and from there it was pretty forgettable really.
"The second phases from set plays are consistently causing us problems and I really don't know where that has come from because we dealt with set plays really well last year.
"We do feel we will always score goals but we do have to tighten up. That is our big objective now.
"It's hard work on the training pitch to put it right. There is no other way and if we take away those second phase set plays we defend ok. But that just shows how much work we have to do in that area."
