Rangers, who have yet to record a Premier League win this season, are propping up the table, a point behind Saturday's visitors.
There are suggestions Hughes could depart if his side fail do not beat Saints - with owner Tony Fernandes reportedly labelling it a 'must-win' game.
Bu the Welshman said: "It doesn't hinge on this match at all.
"People shouldn't under-estimate the changes that have gone on at the club and I have been a big part of that.
"I have had great support from everyone. That doesn't just get swept aside
because we are in a difficult situation."
Hughes claimed Fernandes' comments were taken out of context, but admitted: "I can understand that, because we haven't won enough games - we haven't won any games - and we need to change that quickly.
"My expectation is that we will get the three points tomorrow and we will start the long road back."
On morale within the camp, the Welshman added: "People from the outside maybe have the perception that it's all doom and gloom and everyone's walking around with glum faces.
"But that isn't the case because we know we're very close to turning this around. But we can't keep saying it, we have to do it."