The Football Association have charged Southampton with failing to control their players after referee Mark Clattenburg ruled that Luke Shaw pushed Grant Holt in the area at Carrow Road.
Artur Boruc kept out Holt's resulting spot-kick, but Southampton players continued to protest after the final whistle, with manager Mauricio Pochettino entering the pitch to vent his anger towards Clattenburg.
Hooiveld accepts his team-mates let their frustrations spill over after it appeared they had been denied a crucial point in the battle against relegation.
"If people cross the line, then of course you have to be charged, but I think at that moment we were fighting to keep that clean sheet," said the defender.
"We had fought the whole of what was a tough game and tough circumstances, then they were awarded a penalty which in our eyes was not the right decision.
"Then the emotions went a little bit high and maybe we got a little too close. I think whatever happens after that, it is not up to us to decide."
When asked if Southampton had badly overstepped the mark, he said: "Not really. I think it is also in the moment in which it happened and how it happened.
"Football is a game of emotions and of course it has to be directed in the right way.
"But sometimes if you feel you have been done an injustice as a football player you lose your head a little bit.
"I think that's what happened and what is going to happen, I don't really know."