Westcarr's 80th-minute clincher came out of nothing, but the striker was quickest to react after the U's defence failed to deal with Nicky Shorey's cross from the left, as he prodded the ball home from seven yards.
The result leaves Oxford's new manager Michael Appleton still without a point and reflecting on defeat against his former employers.
In a free-flowing encounter, the home side kept possession well in the first half, and threatened at set-pieces.
Joe Riley saw his fierce 25-yard free-kick in the second minute pushed aside by goalkeeper Paul Jones, and from the corner Jones saved with his legs from Carlton Morris' close-range effort.
Danny Hylton was unlucky with three attempts in quick succession from another corner. Two shots were blocked by defenders and a third came back off the post.
Pompey broke with pace and Miles Storey went close with a low shot that goalkeeper George Long dived on at his near post. Nigel Atangana also stabbed a shot just past the upright.
After Westcarr's breakthrough, Junior Brown had a golden chance to equalise for Oxford, but headed over from six yards.