The Blues boss was sent to the stands after voicing his anger midway through the second half of Saturday's Premier League clash at Stamford Bridge, having felt aggrieved that the visitors were taking too much time over their set-pieces with the scores level at 1-1.
An Eden Hazard double, plus goals from Samuel Eto'o and Oscar sealed the three points for Chelsea, although Holland says Mourinho was still unhappy with referee Anthony Taylor.
"Jose is really frustrated regarding one incident which reoccurred during the first hour of the game and that, quite clearly, from the first couple of minutes, it was taking longer and longer for Cardiff to take their goal kicks, free-kicks, corners," said Holland, standing in for Mourinho in his post-match interview.
"We didn't badger the fourth official; it was a point that was made on two or three occasions quite professionally. Was the referee going to do something about it? It was clearly having an effect on the game, the game was being stopped for long periods.
"The referee did say he was aware of it but nothing actually happened and it continued. I think it was 56 minutes before he finally made the point to the goalkeeper and gave him a yellow card he needed to speed up.
"It was nearly an hour gone by then. The game then changed - we got the goals, then there was an incident near the corner flag, just in front of our dugout. I'm not certain of the reason why, I don't recollect, but Cardiff had to give us the ball back.
"They gave us the ball back, they chose to kick it right down towards the corner flag, which is their right. The young ball-boy went off to receive the ball, threw it back to Ivanovic who was standing in the position it was kicked out - it didn't actually reach Ivanovic, it went towards the dugout.
"It wasn't a very good throw from the ball-boy, and none of this is Ivanovic's fault. He then picked up the ball and within literally two seconds, I'm sure if you watch the footage back, it certainly isn't three seconds, the referee is on to him about getting the ball into play and the game continuing.
"This was really frustrating from all of our points of view given that for an hour, it had been clearly okay to take 20, 25, 30 seconds to take a restart. When suddenly Chelsea had the lead, the new ball had to be thrown back in within two seconds so that was the frustration."
Speaking of the game itself, Holland admitted his relief that Chelsea were able to return from the international break with a win having suffered back-to-back defeats following the last one.
"We were conscious that the last time we came back from the international break, we didn't get the result at Everton and didn't follow that up with the result against Basel," he said.
"It can be difficult. We're very fortunate to have players who play all around Europe at international level.
"The downside of that is that having an international break, you can have a number of players coming back with a selection of issues having travelled here, there and everywhere."
Chelsea had to come from behind to secure the win, with Jordon Mutch opening the scoring with a goal which Holland thought was out of character from his side.
"We really focused on trying to make a good start to the game, the goal was a set-back, it was an uncharacteristic error from our point of view," he said.
"We have been very good defensively of late, but it was just a misunderstanding, a case of the defender thinking that the goalkeeper had it and the goalkeeper thinking the defender had it.
"The response from the players was excellent and to in the end, win the game 4-1 was a great credit to them."