A stunning Peter Whittingham strike had given the hosts a 21st-minute lead at the Cardiff City Stadium on Sunday, but Gray coolly fired home just eight minutes later to level.
Dekel Keinan's first goal for the Bluebirds looked to have won it with seven minutes to play, but Gray's late near-post header dented Cardiff's hopes of a top-two finish.
Craig Bellamy and Jay Bothroyd wasted chances after the break, with the former also furious not to be given a first-half penalty after going down under a challenge from Jason Shackell.
And Cardiff boss Dave Jones will reflect on a missed opportunity following Swansea's defeat at Derby on Saturday as his side remain fourth in the table.
Despite consecutive defeats to Ipswich and Crystal Palace, Jones opted to stick with the same side which lost at Selhurst Park, although Seyi Olofinjana returned on the bench.
Barnsley boss Mark Robins made one change from the XI which claimed a first league win in six games against Scunthorpe, defender Matt Hill replacing Jim O'Brien.
The Tykes were dealt an early blow as Danny Haynes was forced off in the 11th minute with what appeared to be an ankle injury, Gray coming on.
That was the only incident of note during an ugly opening, but the game sprang to life as the visitors took the lead through Whittingham's stunning strike.
The midfielder brought down Jay Emmanuel-Thomas' high flick on the right and lashed an unstoppable left-footed volley beyond the despairing dive of Luke Steele for his sixth goal of the season.
Marlon Harewood had a chance to respond almost immediately after being teed up by Jacob Mellis, but the on-loan Blackpool forward's effort went wide off the outside of the post.
Cardiff then lost Michael Chopra to injury in the 25th minute as the forward went down clutching his hamstring after racing onto a Stephen McPhail pass, Chris Burke coming on to replace him.
And a further blow came four minutes later as a Harewood knock down gave Gray the time to control and slot a controlled strike into the corner beyond Stephen Bywater.
Steele beat out a Bellamy effort following good link work with Whittingham and Emmanuel-Thomas, before Bothroyd blazed narrowly wide from the edge of the box as Cardiff sought a rapid response.
Bellamy then appeared to be dragged down by Shackell as he raced in on goal from a lovely Whittingham pass, but referee Russell Booth waved away the appeals and spoke to the striker after he continued to protest.
Gray should have doubled his tally five minutes into the second half, but he could only head at Bywater after a fantastic ball in from Hill, while Diego Arismendi then glanced a header wide from Kieran Trippier's free-kick.
Cardiff responded and just after the hour mark Steele needed to stick out a leg to block Bothroyd's close-range effort after more good work by Burke.
And the Bluebirds than spurned two gilt-edged chances as Steele denied Bellamy with a sharp stop before, from the resulting corner, substitute Olofinjana put the ball wide.
Steele bravely denied Bellamy again as Cardiff laid siege to the Barnsley goal, and their resistance was broken when a weighted Burke cross was headed home by Keinan.
But the lead did not last long as Gray stooped to head home a Jacob Butterfield cross with two minutes to go.