The hosts trailed 1-0 three minutes from time at Global Energy Stadium but Dutchman De Leeuw headed home an equaliser and then made the injury-time winner for skipper Richard Brittain.
"It is hard to be a substitute all of the time and you just try to do the right things coming off the bench," said De Leeuw. "But today is my lucky day.
"I showed what I came here for and, if this is the catalyst for me to do well for the team, I'm happy.
"It was a nice last five minutes. It is lovely to do something for the group and to do it like this for the team - and for me also - was great.
"I hope this can be the breakthrough for me. I train hard every week, everybody does.
"It was a must-win game today because the other clubs were on eight points and we were on four. Now we are closer to them in the table and it also means we leave Hearts a good distance behind, which is good.
"We can enjoy the win tonight, but then move on to next Saturday and another away game. We hope this is the boost we need.
"It's important to know as a team that you are never beaten. It is never over until the referee's whistle sounds. Before that, everything is possible."
Paterson keen to learn from loss
Hearts forward Callum Paterson said: "We're absolutely gutted. They had pressured us but never really looked like scoring.
"It was just two silly mistakes right near the end of the game that has cost us dearly. It has cost us three points we desperately needed.
"There were numerous occasions we could have scored and put the game to bed. We just didn't take our chances."
However, Paterson thinks the young Hearts side will learn from this disappointment.
"There is never a good time for something like this to happen but we can take a lot from the performance," he added
"It has happened, and we need to learn from our mistakes and make sure it doesn't happen again throughout the season.
"We're learning all the time. Take Jamie Hamill and Jamie MacDonald out of the equation and we have probably played 50-odd first team games between us.
"We're learning in every game and we just need to keep learning."