Scotland had gone ahead twice in the match through James Morrison and Miller himself, but their goals were cancelled out by first Theo Walcott and then Danny Welbeck before debutant Rickie Lambert struck a dream winner.
"We have worked our socks off, given everything we've got, and possibly deserved to get something from the game," Miller told ITV.
"We'll go away and analyse the goals - particularly the second and third goals from set-pieces, we're pretty disappointed with that.
"But overall there's a lot of positives to take from the game.
"If you were to tell us we would come here and lead 1-0 and then 2-1, I think we would have taken that at the start.
"We've come here and put on a decent performance, got two goals at Wembley, but unfortunately it wasn't enough to get anything."
Scotland midfielder Scott Brown reckoned the team gave a good account of themselves in a match both sides were going all out to win.
Brown said: "It was hard work. I think we played really well. And we were obviously devastated to come away with a loss but and the end of the day there are a lot of positives to take.
"Especially in the first half. I think we were the better team. We managed to keep the ball and we created chances as well.
"There's no such thing today as a friendly. I think they knew that and we knew that."