The former Lincoln City striker's 77th-minute volley was a stunning blow for Mark Cooper's Swindon, who had dominated at Gresty Road and were leading through Ryan Harley's fiercely-struck penalty box finish three minutes after the interval.
Swindon had breathed a sigh of relief when Chuks Aneke wasted the chance to give Crewe the lead two minutes before the interval when he lofted his penalty high into the stand.
Swindon were on top in the first-half, but Michael Smith and Nathan Thompson wasted one-on-one chances to ease the promotion-chasing Robins into the driving seat.
Swindon's flowing football kept the home side penned in their own half.
Aneke dragged an early effort wide, but they were mainly on the back foot. Michael Smith, slipped in on goal by Jacob Murphy, delayed his finish and young Crewe goalkeeper Ben Garratt was able to get off his line to make a block.
But Thompson had acres of space to make it count when he was freed by Yaser Kasim's diagonal ball, only for the right-back to skew his shot past the far post.
And when substitute Cameron Park's cross deflected off Harley's arm, Aneke appeared to be ready to give Crewe a morale-boosting interval lead. But the 11-goal leading scorer drove his shot at least five yards over the crossbar and held his head in his hands.
Swindon made the most of their escape within three minutes of the resumption when Harley blasted an unstoppable finish from 15 yards into the top corner after more marauding work by man-of-the-match Byrne.
Massimo Luongo could have doubled the lead, but his first-time effort cleared the frame.
Smith and substitute Danny N'Guessan went close to easing Swindon's passage, but they were caught stone cold by a bombardment triggered by the arrival of Oliver.
Tyrell Belford kept out a far-post header from Aneke, who also hit the bar with a curling drive.
Then Oliver galloped clear and thrashed a volley from 25 yards past the bemused Belford as Crewe secured an invaluable point.