Akpa Akpro was shown a straight red card in the 36th minute for a kick on Frank Simek with Loy pouncing moments later to secure a 1-0 win.
Despite being a man down, Rovers pressed after the break but Carlisle keeper Mark Gillespie denied Adam McGurk and Liam Palmer, the latter also having a goal disallowed for offside.
Gillespie was forced into his first save early on, thwarting David Amoo when one-on-one before Akpa-Akpro lashed out at Simek as the pair scuffled near the touchline.
Loy bundled home the opener for Carlisle in the 44th minute while after the break Gillespie made a fine one-handed save after McGurk had latched onto Zoumana Bakayogo's cross.
Matty Robson cleared an effort from Tranmere defender Donervon Daniels off the line while Gillespie needed both hands to beat away a stinging Palmer volley.
And Rovers' misery was compounded in the 71st minute as they had a goal chalked off for offside after Palmer had poked home Andy Robinson's free-kick.