The home supporters were incensed seconds before the break when Troy Deeney, seemingly through on goal, was hauled back as he lined up the shot but the referee waved play on.
And Zola was visibly frustrated about the decision, he said: "It was one of those days that has been made like it is by one man on the pitch today - and he wasn't wearing a yellow or an orange shirt.
"I would love to spend more of my time talking about somebody else and I don't like talking about the referees.
"But today he's made a difference and I feel so upset, not because of him but because of my players, because they didn't deserve to lose a game like this.
"They worked very hard and they deserved something more on the pitch. The reason they didn't get it was because of him.
"He had a very poor day and he had two opportunities. He had the opportunity towards the end to at least make up a little bit for his bad day and he didn't do that and I feel furious about it."
Despite the defeat, the Hornets boss remains confident about his side's promotion prospects.
He added: "Today was very difficult circumstances, with a lot of injuries and players that just played for the first time and players that hadn't played for a long time.
"And we played a good game against a very good team, so I'm more than convinced that we'll be fighting to the end."