The Glovers' promotion hopes suffered a blow after goals from Billy Clifford and Freddie Sears gave the U's a surprise win.
Johnson said: "I don't think we deserved anything from the game. We didn't really show any of the innovation we've shown in our good run.
"Colchester got into our faces and shut us down early and when they went ahead, it was more power to their elbow at that point.
"It gave them the strength to keep going and although we huffed and puffed, we didn't get anywhere. It was a long way for our supporters to come and we didn't really give them anything.
"The players are disappointed, but we looked weak and we haven't been like that.
"We've still got a lot more points than Colchester have and we have to quickly realise that we're still up there challenging.
"But it was a poor performance and you have to hold your hands up.
"There was nothing out there that we could hang our hat on and we gave their goalkeeper Sam Walker a comfortable night."
