Billy McKay's opener was cancelled out by a brace from Josh Magennis but Gary Warren drew the away side level before McKay's second goal won the game for Inverness.
"I'm disappointed for the fans again. In our last four games we've had two at home and two away and we've won the two away games, which isn't going to please the support," said Brown.
"They came out in good numbers on a terrible night, and we get ourselves into a good position and then lose two soft goals, which is unacceptable."
The Aberdeen fans were angry the Inverness opener was allowed to stand and Brown agreed with their view that Jamie Masson had been fouled.
"I thought the opening goal was dubious," he added. "The second came from a corner we didn't need to concede, and then the third we were deficient defensively.
"Jamie Langfield didn't have much to do, but we've lost three goals."
Aberdeen could have gone top of the SPL table if they had won and Brown was left to consider what might have been had more of his senior players been available to him.
"I didn't say to the players about going top but there's an expectation the fans have, and I do wonder if the experienced players we were missing would have been able to withstand the expectation," he said.