The Foxes looked on course for victory when Ben Marshall swept them ahead in the 52nd minute but two spectacular late goals from Davide Petrucci and Grant McCann secured three points for the Championship's basement side.
Ferguson said: "It has been a very good day to get that performance and result. We were the better team from the off and scored a couple of outstanding goals to get the win. I can certainly go home and enjoy the rest of my birthday.
"It is also the first time we have come from behind to win a game all season and it was about time we produced a fightback like that.
"We've always had the belief in the dressing room that we can stay up and hopefully a victory like this will give that belief to the fans.
"If we can repeat this level of performance on a consistent basis we will have a really good chance.
"The only disappointing thing is that we are without a game next weekend."