Will Evans put Hereford ahead in the third minute and Ryan Bowman netted a goal in either half after Luke Summerfield's free-kick on the half-hour mark had made it 2-1.
Turner, who spent 15 years at Hereford before leaving in 2010, said: "It's difficult to explain why we go wrong in the early stages of matches.
"This is the fourth time in five games that we have conceded in the first five minutes and then the second goal just compounded the problems.
"You cannot give teams a start, no matter what level they are at. It lifts the crowd, it lifts the players, we go down a notch or two and we were chasing the game after that.
"We contributed to Hereford's victory, there is no disguising that - they worked hard for their win but we are out of the FA Cup and it's devastating."
