The substitute winger was sent off for a head butt 12 minutes from time, having joined the action on the hour mark.
Ben Pringle fired the hosts ahead, with Rhys McCabe levelling matters for the Owls before Lee Frecklington grabbed Rotherham's second seven minutes before half-time.
And Jones said of Johnson: "It was just stupidity, you can't headbutt somebody, it's just not acceptable.
"He's been told I can't defend it and neither can JJ.
"We're trying to get back into the game, it's hard enough with 11 men and going down to 10 makes it more difficult. He'll be punished and rightly so because I lose him for three games."
As for the match, he added: "What cost me the game is we didn't defend two long throws into the box.
"I was disappointed we didn't get a grip of the game from the word go but we knew it was going to be difficult.
"I thought there was a bit of a panic with us for clearances. It's disappointing because we know we are better than that."
Delighted Rotherham boss Steve Evans added: "It was a great win for us against a great football club.
"We always believed we could win but we knew we would have to play well.
"We did our homework and we had a game plan. It was a magnificent cup tie and it was a great achievement for this football club to win at home for the first time against Sheffield Wednesday for 37 years."