At 12.30pm on the fourth day of the first Test, the former Black Caps wicket-keeper felt Brendon McCullum's side were in good shape after they were set a target of 239 to win the game.
But by 2.30pm they had been skittled for just 68 runs, as James Anderson and, in particular, Stuart Broad, turned on the style for England.
And Smith said this defeat will hurt more than the losses they suffered in South Africa because they were so competitive in the first three-and-a-half days at the Home of Cricket.
He told Sky Sports: "I'm a little bit shell-shocked by what unfolded in the last couple of hours because up until that point they played the equal of England in the game, perhaps even a little bit better in a lot of areas.
"It all just turned so poor so quickly. I don't know whether you sit back and reflect too much on a performance like that because it can perhaps do you damage.
"[The defeat in] South Africa was consistently on the back of some overall bad performances, never being in the game, but they were in the game for a long time here.
"The thing that will hurt them most when they sit down and reflect as individuals on what they could have done better is that the underlying factor will be we could have won.
"We could have won this game because we played well enough for most of it."
Smith says New Zealand's batsmen have a soft underbelly that can too readily be exposed by consistent seam bowling.
And he said the form of Broad, who bagged seven wickets in just 11 overs on Sunday afternoon, was too much for the tourists to handle.
"They put it in the right areas more often than perhaps we expected," he added.
"We came to the ground looking at Swann and all these stats about spin bowling and he never got close to having a bowl. That's how quickly it all unfolded and how unpredictably stupid this game can be at times.
"There is a vulnerability about this New Zealand side and the underbelly was again exposed, but I think their mindset was to play like we play.
"There's enough strikers and enough positive players in that side to suggest if they'd got one partnership or one-and-a-half partnerships together of note at a decent rate then Alastair Cook might have had something to think about.
"There's a quality of consistency in the bowling here. That was good bowling; there was some average judgment at times outside off-stump, but there were some damned good deliveries and consistent stuff.
"Like England, they have been batting on those good pitches [in New Zealand] remember and had their way with the English attack for quite some time.
"Not this time around."