The home side took the lead through a deflected Steve Sidwell shot only for Hatem Ben Arfa to equalise with a superb 20-yard shot after 54 minutes.
But Newcastle were only back on level terms for nine minutes before Hugo Rodallega scored the winner when he headed home a Damien Duff free-kick as Pardew's side failed once again to record an away victory in the Premier League.
"It's a really soft second goal - a soft free-kick, soft defending on the free-kick and you can't do that at this level because we had the game by the scruff of the neck at this point," said Pardew.
"We played some of our best attacking football of the season, but we've still come away with nothing. Performances don't get you points.
"In the second half we were terrific other than those two or three moments. We've got no-one to blame but ourselves."
"I think we've conceded poor goals at crucial times and away from home it's costly."
Newcastle's next game is at home to defending champions Manchester City but Pardew believes his side can still challenge the top sides at St James' Park.
"I think at home we are a threat to anybody," he added.