Dutch forward Marvin Emnes, who signed a three-year contract extension this week to end speculation linking him with Premier League Swansea, put Boro ahead midway through the first half, only for David Norris - another Pompey new boy - to level shortly after the interval.
Rhys Williams quickly restored the hosts' advantage after being played through by Nicky Bailey, but the hosts were unable to find a killer third goal and paid the price when Varney fired home in the second minute of added time.
The visitors will be delighted to have avoided a fourth successive opening-day defeat, having seen their opponents create a string of chances in difficult conditions.
Carl Ikeme made his debut in goal for Middlesbrough following his loan move from Wolves, while visiting boss Steve Cotterill, who named just four substitutes, handed first competitive starts to summer signings Norris and Varney.
Boro were the first to threaten as Barry Robson fired over from distance.
Former Middlesbrough loanee Dave Kitson then tested Ikeme at the other end before Robson spurned a clear-cut opportunity in the 18th minute.
Tony McMahon's free-kick picked out the former Celtic midfielder 10 yards from goal, but he directed his header straight at Jamie Ashdown.
The miss appeared unlikely to be costly as Tony Mowbray's side took the lead through Emnes six minutes later.
Strike partner Scott McDonald found his way to the byline before cutting the ball back to the Dutchman, who took a touch before swivelling and drilling a low shot into the bottom left corner, possibly with the aid of a slight deflection.
Emnes shot narrowly wide soon after as the rain grew heavier, while McDonald, Justin Hoyte and Robson also went close as Boro threatened to extend their advantage.
Pompey had a chance to draw level a minute before the break, but Hayden Mullins failed to keep his shot down after good work from Tal Ben-Haim.
It took the visitors less than two minutes to get on the scoresheet in the second half as Norris beat Ikeme with a powerful low shot from 20 yards.
However, Boro were soon back ahead as Williams collected a glorious through ball from Bailey before finishing coolly.
The goal once again led to a spell of home pressure, but the Teessiders failed to extend their lead.
Cotterill introduced veteran forward Kanu and Ryan Williams, the younger brother of Middlesbrough goalscorer Rhys, in a bid to find an equaliser, while Christian Dailly was also brought on with seven minutes remaining.
Emnes almost grabbed his second of the game moments later, but Ashdown reacted well to tip his shot over the bar.
Varney then popped up to shock the home support after Dailly's initial effort had been blocked.