Little- making only his second start this term - handed the holders a first half lead at Rugby Park after Kilmarnock had an early effort ruled offside.
Steven Naismith then continued his fine form this season when he netted in the second half to put the quarter-final tie beyond the Ayrshire side.
Youngster Gregg Wylde made his first start for Rangers as they made five changes from the Old Firm derby. Goalkeeper Allan McGregor was swapped for Neil Alexander, while Vladimir Weiss, Kirk Broadfoot, Little and Wylde were also drafted into the side.
But there was no place for the luckless Andy Webster, who had been expected to make only his second appearance of the season but missed out amid suggestions he was injured.
Kilmarnock made just one change from their defeat to Inverness at the weekend, with Mehdi Taouil starting ahead of David Silva, who was named on the bench.
Kilmarnock posed more of a threat in the opening spell and thought they had taken the lead when they had the ball in the back of the net after just seven minutes.
Alexei Eremenko won a free-kick just outside the box and dusted himself off to play the ball short to Craig Bryson who flighted into the box where Liam Kelly was waiting to nod home but the effort was ruled offside.
Rangers had a decent chance of their own when Wylde's corner found David Weir at the back post but the skipper was denied the opener when his header was smothered by goalkeeper Cammy Bell.
Ben Gordon then lost possession to Naismith in a dangerous area and he set up Little for the shot only for the Northern Ireland international to fire across goal and well wide of target.
The Rangers pair combined well again and had more luck this time when Naismith nodded on for Little and he sent a superb half-volley into the top corner of the net to hand the holders the advantage with 25 minutes gone.
Killie could have drawn level when Kelly slipped into the path of Jamie Hamill and he squared across goal for Taouil but the danger was cleared by Weir before he could unleash the shot.
Conor Sammon then raced past Steven Whittaker into the box only to have the ball snatched from his feet by Weir as he, too, was denied the chance to even trouble Alexander in the Rangers goal.
However, the goalkeeper was called into action to confidently deal with a Manuel Pascali header just before the break.
With just one goal in the tie, Kilmarnock emerged for the second half determined to haul themselves back into the game. Sammon tried his luck with an ambitious shot from distance that took a deflection and fell just wide of the upright.
Alexander then produced a superb save to tip Pascali's header just over the crossbar, before the same player saw his shot fizz past the post from the resultant corner.
But it was Rangers who left the back of the net bulging after 60 minutes when Wylde's pass was collected by Naismith and he dispatched a low shot past Bell into the bottom corner.
The home side could have pulled a goal back soon after but Kelly could only look on in disbelief as his shot cracked off the upright and the chance was lost.
The last thing Kilmarnock wanted to see was the sight of Rangers' top goalscorer Kenny Miller being thrown into the action, along with Kyle Hutton, for the last 20 minutes.
But it was Mixu Paatelainen's men who could have found the net when substitute Rui Miguel squared for Sammon with the goal at his mercy but his shot crashed off the turf before bouncing just over.
Rangers were able to see out the rest of the clash at Rugby Park to ensure their names would be in Friday's draw for the semi-finals.