Magath's team began the day bottom of the table, but they moved up to 18th place thanks to Colombia striker Hugo Rodallega's winner with four minutes remaining at Villa Park.
Kieran Richardson had put Fulham ahead just after the hour before Grant Holt equalised with his first goal for Villa nine minutes later.
Although Fulham remain five points from safety, the result will give them renewed hope that a great escape is possible.
"Yeah, we always believed we would win," the German told Sky Sports.
"Today, for me it was not only a win because we went in front and they got the equaliser. In these situations in the last few games we've gone down, but today we fought back and got the win, so I'm very happy and proud.
"We played with the confidence that we are capable of producing and can stay in the league. The win gives us the right atmosphere (in the camp) to go on and to fight for Premier League safety.
Magath brought on Ashkan Dejagah and Rodallega at the break and it was the Colombian, who netted late on to secure three vital points.
"Our plan was to begin a little bit more defensive and in the second half I wanted to be a lot more attacking," added the former Wolfsburg head coach. "Hugo was in very good shape today. Not only did he score, but he was all over the park - such a flexible attacker."