During his three years with Norwich, Lambert took his squad from League One to the top flight before moving to Villa last summer on acrimonious terms.
On his return to Carrow Road earlier this season Villa won 4-1 in the Capital One Cup, and Lambert is calling for a similar performance as he looks to secure his team's Premier League status.
Lambert said: "I think if you get caught up in my connection there it deflects from the game. It's not about me going back there because I've been back already.
"I had three fabulous years there and took a team from League One, built it up, the crowd bought into it, the lads I worked with were fantastic.
"It's another game and I'm looking forward to going down there. I have fantastic memories there but it has gone.
"The players I worked with were fantastic and one or two are still there. I probably know them better than they know themselves, really.
"There's a few teams right in the relegation battle anyway. We know we've got a hard game there but we're certainly playing well enough. That's the big difference.
"My job is here. We have to win. If our fans are anything like they were for the cup game then we'll be taking a ferocious support down there. It will be a tough game but we're certainly playing well."