The Terriers produced an excellent 2-1 win at Middlesbrough in midweek but were off the pace at the John Smith's Stadium - Callum Harriott's fourth-minute effort bagging the points for Charlton.
"What a difference a couple of days make," said Robins afterwards. "We were brilliant on Tuesday but today we have been undone by a set-play.
"It feels like a step forward then two back but we have to take these on the chin I am afraid.
"We've not had enough quality. We have had plenty of ball in their box and although the way we got it there wasn't ideal, needs must at times."
Robins was also unhappy with the pitch, adding: "The pitch wasn't brilliant again. You struggle to do things quickly enough and when Charlton play a counter-attack, they will get people close to you and break against you quick, you are always vulnerable. We've been lacklustre."