The mercurial Italian took over in charge towards the end of last season and successfully guided the Black Cats to Premier League survival.
However, he is renowned for his fiery personality and received some criticism for his no-nonsense handling of players and his strict training methods.
But, asked whether his players feared him, Di Canio responded: "The players cannot fear the manager, it's not possible. I don't kill people, I let them train!
"No, they are not worried. Obviously, they don't love me, and I told them they don't need to love me, they have to follow me, and then they're going to be OK.
"Sometimes we have a fight and sometimes they drive me crazy, and then I'm going to get angry, but to be honest it is more like a family, with respect.
"Sometimes the son or the daughter goes away from the right road and I pull them back."
Di Canio was keen to improve the fitness of his squad towards the end of last season and still thinks that is an area he needs to focus on.
"Our team is a very good team, but not a top team," he said. "To reduce the gap between us and the top clubs it's obvious that we need a strategy and tactics and technical quality.
"But physical condition is also one of the main parts, otherwise it is difficult to reduce this gap.
"We train in a different way to other clubs and have different things to improve. Athleticism is crucial to competing at the top level."
Di Canio was speaking at a press conference in Hong Kong, where Sunderland will play Manchester City on Saturday in the final of the Barclays Asia Trophy.