A scrappy contest erupted after 77 minutes when Pompey skipper Johnny Ertl tangled with Anthony Grant on the ground, before Yassin Moutaouakil aimed a head butt at the Boro man.
Midfielder Moutaouakil got away with just a yellow card before home boss Guy Whittingham promptly substituted him and Westley hauled off Grant.
Westley said: "The referee can only give what he sees and he has obviously given what he has seen. I'm sure if he thought there was a head butt, he would have sent the lad off.
"From where I was standing, it looked like a clear red card. The fourth official, I think, felt the same."
The Boro boss was happy with Stevenage's resilience on the south coast.
He added: "That's a second clean sheet in a week and it was a hard-earned clean sheet.
"We could have easily scored two or three goals from pretty clear-cut chances and we didn't give too many away. On the whole, I think a draw was a fair result in the game."