Pulis felt his side 'shot themselves in the foot' as they took the lead through a Jon Walters penalty before losing the match to strikes from Yohan Cabaye and Papiss Cisse.
"We are really disappointed and we shot ourselves in the foot, the first goal we gave them - it is a dreadful mistake from Glenn (Whelan) and he has held his hand up and they have scored a free-kick from that," he told Sky Sports.
Cisse - who scored the winner - was booked along with Marc Wilson for an altercation between the pair, and Pulis thinks the African should have been sent off after he shoved both the Irishman and Ryan Shawcross in the aftermath.
"The second, I am not so sure Papiss Cisse should have been on the pitch, I know he scores but he raised his hands and caught Marc Wilson and Ryan Shawcross," said Pulis.
"But that is the way it is going for us and we are really disappointed and sums up our season away from home, where we have played really well at times and should have taken points and this is another game we have taken none from.
"Newcastle had some good play in front of us and to the side, but aside from the goals I don't think Asmir [Begovic] has had a save to make. I know that is football.
"I am very disappointed for the lads because they have put such an effort in today and away from home this year I think we have played better than any time in the Premier League. But we have just not had the results and we have shot ourselves in the foot, it is not Newcastle it is our bad defending or mistakes that have cost us the result."
Pulis admits that Stoke - who are now level on points with Newcastle - are way behind where they should be.
"We should be a lot more points better off, no question of a doubt and that is disappointing but we have got good spirit, good characters in the group and hopefully we can bounce back," he said.