Loy bundled home what proved to be the winner a minute before half-time and eight minutes after Jean-Louis Akpa Akpro saw red for kicking Carlisle defender Frank Simek.
Tranmere threw everything at the Cumbrians after the break but found keeper Mark Gillespie in inspired form while Liam Palmer had a goal ruled out for offside.
And, after falling to a 3-0 defeat at the hands of the League One leaders in their third game of the season, Abbott insisted it felt good to get his own back.
"Tranmere got our season off to an absolute nightmare when we played them in our first home game so we got a little bit of respect back against a side that has been unbelievably good this year," said Abbott.
"We are happy bunnies but there is a long way to go and we have our own ambitions to move up the table. Unless you roll your sleeves up you will get turned over at Tranmere.
"They work and fight for everything but we have got an honest and determined group of staff and players and I think we just about deserved to win.
"The sending-off probably helped them more than it helped us as we became a bit complacent but we have won the game and that is all I care about.
"Our defence has to take a lot of credit because we came under a lot of pressure and they have come under scrutiny at times this season but not too many teams come to Tranmere and get clean sheets."
