Martin Paterson came off the bench to salvage a point in the 85th minute, after Tommy Rowe had netted twice for Posh in the space of four second-half minutes.
Charlie Austin had given the Clarets the lead inside 65 seconds and Dyche said: "It's a pleasing point in one respect as it takes us closer to the play-off places.
"But it's also a frustrating result as we felt this was a game we could have won.
"We enjoyed a fantastic start and controlled the first half of the game on a very awkward surface.
"Peterborough had a real go in the second half, as you would expect from a team fighting for their lives at the bottom of the table.
"We conceded a poor goal to be pegged back and then found ourselves behind, but our boys always keep going right to the bitter end.
"We're four points off the top six and in good shape but we're hungry for more than that."