Kee grabbed his ninth goal of the season just when it seemed the Exiles would hold on for a point, coming after the Brewers' narrow success over Northampton on Boxing Day.
Defender Phil Edwards proved a handful for the Newport defence as Burton dominated the opening half.
The full-back had a header saved by Lenny Pidgeley in the fifth minute and then volleyed an Alex MacDonald corner against the underside of the crossbar.
With Burton well on top, Shane Cansdell-Sherriff's header thudded into the body of Pidgeley at point-blank range just after the half-hour mark and the goalkeeper also reacted well when Kee seemed certain to open the scoring.
Michael Flynn went close with a 25-yard effort as the visitors showed more attacking intent, but they had to rely on Pidgeley again as he stretched to turn a Kee header over the bar.
The visitors got another lucky break in the 61st minute when Adam McGurk's deflected shot hit a post with Pidgeley, for once, beaten.
But Kee popped up with the winner 10 minutes from time when he ghosted in behind the Newport defence to convert a McGurk cross.