Former Stockport and Leeds forward Elding, a free summer signing from Kettering, headed home against the run of play early in the second half at Oakwell on Tuesday night to stun Mark Robins' Championship hosts, who crashed to a 4-0 defeat at QPR in their opening league game on Saturday.
Barnsley midfielder Jacob Butterfield had numerous first-half chances, but failed to break the deadlock when he pulled a shot from the edge of the penalty area wide just before the break.
The young midfielder fired over the crossbar with an early effort, saw another shot held by Rochdale goalkeeper Josh Lillis and was inches off target with a header in the 38th minute.
Lillis denied Adam Hammill as Barnsley picked up where they had left off after the restart, clutching onto another long-range drive after team-mate Nathan Doyle had skewed off target.
Barnsley manager Robins then replaced both Butterfield and Goran Lovre with Martin Devaney and Iain Hume respectively and Rochdale took advantage of the reshuffle by snatching an unlikely 56th-minute lead.
Joe Thompson swung over a cross from the right and Elding stole in to head the ball into the bottom corner.
The hosts battled to get back on terms, but took over quarter of an hour to muster their next effort on goal when another shot from the edge of the area by Hammill was again held by Lillis.
Hume and Devaney both missed the target from inside the area as Barnsley rallied and Liam Dickinson, on his Tykes debut, contrived to miss from six yards following Jay McEveley's pin-point cross as Dale held on for a shock win.