Former Stevenage boss Westley left his position at Deepdale on Wednesday after a run of seven matches without a win saw the Lilywhites slip to 17th in League One.
After first-team coach Dino Maamria also left the club with immediate effect, North End have given Dreyer the added responsibility until they find a new manager.
Dreyer's first game will be Saturday's home clash with leaders Bournemouth.
A club statement read: "The club would like to place on record its thanks to Dino for his efforts whilst at Deepdale and wish him all the best for the future.
"John Dreyer took charge of the Lilywhites for training on Thursday and will be in charge of team selection for the clash with Eddie Howe's table-toppers and until a new manager is appointed.
"The club will now begin its process to find a permanent replacement for Graham Westley and will not make any further announcement until this process is complete."
