Kieran Trippier's fifth-minute free-kick and Danny Ings' fine run and finish just after half an hour saw Burnley safely through into the third round of the competition.
However after the break Preston forced Burnley goalkeeper Alex Cisak into a number of fine saves while Jason Shackell also had to clear a Stuart Beavon header off the line, leaving North End manager Grayson to cut a frustrated figure.
"I am disappointed to go out of the cup but I am very proud of the players," he said.
"Each of them played to a high standard and the scoreline doesn't reflect the opportunities we had at times.
"The goals we conceded were both avoidable and probably came against the run of play.
"But over the course of the 90 minutes we were the dominant team and unfortunate not to get something more from the game.
"We didn't have the ideal start but we showed a lot of character. We passed the ball well and worked extremely hard.
"Their goalkeeper has made some great saves and we caused Burnley a lot of problems.
"Our supporters will have left disappointed to go out of the competition but happy with the fact that the players have played for the shirt.
"Nobody likes losing games but we lost in the right manner because of the way we performed and the work ethic we showed.
"We had to keep believing in what we were doing and if the goal had come in the second half I think we could have gone on to win it because the momentum was with us."
And Burnley manager Sean Dyche backed up Grayson's claims after admitting his side had been given a run for their money.
"We never expected an easy game and we had to work hard," Dyche said.
"In a strange sort of way us scoring early gave Preston a sense of freedom because they had to come out and have a go.
"They did that but our second goal calmed things down a bit although after the break they really stepped onto us and stopped us from playing for a while.
"Then we gathered our thoughts and began to use the ball better as the half went on.
"I think we created enough good quality chances overall to deserve to win the game which is what will happen if you put them away.
"We had a few near-misses and saw one cleared off the line and although we are still a work in progress I was reasonably pleased with the performance.
"It is very difficult to dominate a whole game and you have to find different ways of winning a match."