The Saddlers were 2-1 ahead and in control against Exeter at the weekend until defender Manny Smith saw red in the 66th minute - a decision right-back Taundry felt was unjust.
The Grecians hit back to triumph 4-2 and leave Walsall just three points ahead of the drop zone, albeit knowing victory against the Terriers this weekend will be enough due to their considerably better goal difference.
"We looked comfortable with 11 men, especially in the first half. And then things went to pot in the second half when Manny got sent off, our shape changed and everything," Taundry told Saddlers Player.
"We just need to have a look at ourselves and produce a win next week."
He added: "It's squeaky bum time. It's not a nice position, nobody in football likes to be at the bottom of the league. Obviously, we play Huddersfield and we'll look to get the three points and see us safe."
