The talented Lolley, who is expected to complete a move to Huddersfield by the end of the week, lived up to his billing with the winning strike.
The deadlock was broken on 26 minutes when Posh skipper Tommy Rowe got the final touch to force the ball over the line.
The visitors drew level on three minutes after the break when Michael Gash met a left-wing centre to plant a firm header past Bobby Olejnik from close range.
Mark Little hit the post just a minute later, but a swift counter-attack, led by Lolley was emphatically finished by Jack Byrne to make it 2-1 to the visitors.
Posh levelled after Lee Vaughan needlessly felled Britt Assombalonga (74) in the box and he stepped up to send the keeper the wrong way from the spot.
However, within two minutes, Kidderminster were back in front as winger Lolley cut inside from the left and drilled a left-footed shot past Olejnik and into the back of the net to send the visitors through.