Both sides took time to come to terms with the windy conditions at the Tulloch Caledonian Stadium but the pendulum swung the visitors' way when the Greece striker fired high past Jonny Tuffey in the 33rd minute to give them the interval lead.
In the 68th minute, after Caley defender David Proctor had conceded a penalty for a foul on Gary Hooper, Celtic skipper Brown slotted in the spot-kick for his third goal in three games to take the holders into Monday's draw at Hampden.
The result extended the Hoops' domestic winning run to 15 games and, being one point clear of Rangers at the top of the Scottish Premier League and in the final of the Scottish League Cup final, kept them on course for the domestic treble.
Once again Celtic manager Neil Lennon tinkered with his side, making four changes.
Defenders Adam Matthews and Kelvin Wilson, midfielder Kris Commons and striker Samaras came in for Cha Du-ri, Emilio Izaguirre, James Forrest and Anthony Stokes, the latter two starting on the bench.
Caley boss Terry Butcher replaced keeper Ryan Esson with Tuffey and brought in midfielder Graeme Shinnie for Shane Sutherland but they never looked like replicating their two previous cup shocks over Celtic in the tournament, in 2000 and 2003.
Inverness were unbeaten in seven matches but on a freezing, windy day they struggled as much as Celtic to tame the conditions.
In the sixth minute Hooper managed to escape the attention of the Caley defence but he was unable to get the ball under control and ended up with a back-heeled effort which was blocked by Tuffey.
It quickly turned into a scrappy affair. Celtic keeper Fraser Forster was called into action for the first time in the 23rd minute when he had to dive smartly down to his right to tip a low drive from Nick Ross round the post.
Caley defender Josh Meekings flicked on Greg Tansey's corner at the near post but the ball was blocked by sheer numbers inside the six-yard box.
On the half-hour mark, as Celtic tested the nervy looking Highland defence again, Hooper had a decent effort from the edge of the box saved by Tuffey.
However, the cup holders' opener was only three minutes away with Tokely at fault.
The big Inverness stopper stretched a leg out but failed to cut out Commons' through-ball and the Greece striker raced on before firing high past Tuffey from 16 yards, with the ball appearing to take a swerve.
Moments later, a short pass-back from Tokely allowed Hooper to run in on goal but Tuffey blocked his angled shot with his leg and from the rebound the keeper saved again, this time from Joe Ledley before the danger was eventually cleared.
The home side were all over the place but six minutes from the break Tokely headed a Jonny Hayes corner over the bar, passing up on the chance to make amends for earlier indiscretions.
Celtic looked in command by the break, though, and they came out for the second period determined to get the next goal.
In the 50th minute, following a fine move involving Commons and Hooper inside the Caley box, the former Scunthorpe striker cut the ball across goal only to see Ledley clear the bar from close range.
Then the sliding Samaras could not get enough on a great Ledley cross from the left and the ball spun past the far post for a goal-kick.
On the hour mark, with the home side starting to make some progress, Sutherland replaced Tade and Claude Gnakpa came on for Ross, before Forrest replaced Commons.
Samaras came close again when he reached a Charlie Mulgrew free-kick with a flashing header which drew another good save from Tuffey.
However, the tie was effectively over when Brown grabbed number two from the spot - the goal again coming as a result of careless Caley defending.
When Hooper robbed Proctor on the byline, the Inverness defender chopped him down leaving referee no option but to award a penalty from which the Hoops skipper sent Tuffey the wrong way.
Forrest finished a driving run into the box with a cross-cum-shot which just evaded Hooper at the back post - Celtic's last real chance before they cruised to the final whistle.