The Millers smashed in four goals during 14 first-half minutes to stun the home support at the Kassam.
"We had a mad 15 minutes," Wilder said. "We're desperately disappointed, we've let a lot of people down.
"That's the biggest defeat I've had at home in my time as manager here and it hurts.
"But Rotherham have got Championship players. And if you give Championship players opportunities, they will punish you.
"I take responsibility for changing the system, but it wasn't the system that caused us to concede the goals we did.
"Looking at the bigger picture, we've played the three top sides in our league, in my opinion, and won two of the games.
"But we were nowhere near for that mad 15 or 20 minutes. We got a little bit of reaction in the second half.
"There was a lot of emotion out there with people (fans) leaving before the end, but we have to look at the bigger picture."
