Jason Crowe's first goal for Leeds seemed destined to rescue his team from defeat until Anyinsah headed home Ian Harte's 84th-minute free-kick to restore Carlisle's advantage.
Kavanagh showed finishing of the highest quality to gain a 21st-minute lead. Richard Keogh headed down Evan Horwood's deep cross for Carlisle's 36-year-old Republic of Ireland international to turn and volley a stinging right-foot finish from 10 yards.
In contrast, Jermaine Beckford, who voluntarily came off Leeds' transfer list at the weekend, was guilty of four first-half misses and he hit a post.
With Kavanagh pulling the strings in a free role, Carlisle were always a threat.
Leeds' improvement after the break was rewarded in the 56th minute when Crowe headed in Andrew Hughes' cross at the far post but Anyinsah had the final say by punishing slack marking.