Javier Hernandez scored the only goal of an absorbing game as United responded in the best way possible to their 4-1 mauling by neighbours City on Sunday.
Evans was making his first start of the season and he made no bones about United's desire to lift the cup.
He told Sky Sports 1: "We want to win it. It was the first trophy I won and it was a great feeling to win it then.
"It's a competition we want to win, we will be taking it seriously. I think the fans enjoyed it, we certainly enjoyed it.
"It was a hard game, Liverpool have been playing expansive football. I am just glad we got the goal after half-time and held on.
"They put out a strong team. There were a lot of lads out there who haven't had a lot of football and were getting a bit tired towards the end."
Fellow defender Chris Smalling admitted the victory was doubly important following Sunday's thrashing by City.
"It was a massive turnaround," Smalling said.
"It was a big stage for us to go out there. There was a lot of doubt at the weekend, we needed to go out and put on a performance."
For the England defender, the intensity of the game was reminiscent of the Premier League.
He said: "I felt like it was end to end for pretty much 90 minutes. It shows we can go out there on the big occasion and produce."