Sloppy defending from the hosts allowed Stuart Fleetwood to give the Bulls a ninth-minute lead on Saturday when he had time to control the ball and shoot past Nicky Weaver from six yards out after meeting Jake Jervis' low cross.
But Darren Potter struck Wednesday's equaliser six minutes later with a firm drive from just outside the area after the Hereford defence had failed to cut out Daniel Jones' low cross.
Clinton Morrison made it 2-1 to Wednesday from the penalty spot in the 70th minute after he had been hauled down in the area by Joe Heath, who was sent-off for the offence.
A fine individual effort from Jermaine Johnson put the home side further ahead seven minutes later when he latched on to Weaver's long ball and finished expertly.
Two minutes later, Morrison found the net from the spot again after Michael Townsend was ruled to have handled inside area.