Manager Davies, who was forced to watch from the stands due to a five-match touchline ban for abusing a referee, saw his promotion-chasing team lose for the third successive game following Dale Jennings' 51st-minute winner.
Davies was also not allowed in the dressing room at half-time or 30 minutes either side of the match because of the ban.
Kelly said: "It's difficult with the gaffer not being there, it's different with the gaffer not being there because he's the leader, he's the focal point of the team.
"Communication was a little bit difficult with the phones; phones don't really work in football grounds. But we had conversations going on.
"It's not the same when he is not there, we miss him. We miss him as a staff, we miss him as a playing group but such is the ban."
Wideman Jennings fired into the bottom left-hand corner from the edge of the area for his first Tykes goal to lift his side off the bottom, two points from the safety line.
Forest, who remain in fifth place after the narrow defeat, went closest when Algerian winger Djamel Abdoun hit a first-half shot which thumped off the base of the right-hand post.
Kelly added: "We just couldn't get the elusive goal, we tried to change the formation, trying to get it forward a little bit quicker.
"I'm just a little bit disappointed we couldn't break them down, they defended for their lives; it was a cup final atmosphere, the crowd was absolutely jumping so I'm just disappointed really.
"We deserved at least a point, if not all three. You need a bit of fortune sometimes, especially when it's a bit of a bobbly pitch.
"I can't criticse the players at all. The gaffer worked on things on Thursday and Friday for the specifics of the game today and we did everything except score the goal that was required."
Barnsley boss Danny Wilson dedicated the victory to the fans after last week's 5-0 thrashing at Huddersfield.
"It was not only the win that was important to us today, I think the performance and giving something back to the fans today was a major, major priority for us," he said.
"We didn't see the true us last week, over the last few months we've showed better resilience than we did last week but I think we went a long way to putting it right today.
"We responded very, very well and I have to say the fans were as good as we were today; absolutely top class, getting behind us.
"I think the last 20 minutes was a little bit nervy but the fans didn't show the nerves, they just got behind us and drove us through it, so it was a great team effort from both the fans and the players today.
"At the end, when they're throwing everything at us, we had to defend resolutely and I think everybody did, not just the back four.
"But I will say again and I want to emphasise it; the fans were absolutely brilliant and that was a great atmosphere at the end."