Against the nation ranked 58 in the world, Mario Gomez gave Joachim Low's side the lead just before the break in Leipzig.
A late goal from Andre Schurrle earned Germany a slightly more respectable margin of victory, but after losing 5-3 to Switzerland last weekend, Low still has a lot of fine-tuning to do before things get serious on June 9.
Low selected seven of the eight Bayern Munich players who joined the squad on Sunday, leaving only the injured Bastian Schweinsteiger out.
One of those, defender Jerome Boateng, was almost the unlikely provider of the first goal when he rattled the post with a shot in the 19th minute of what was already becoming a one-sided affair.
The opening goal came from a more predictable source with Gomez firing in off the underside of the crossbar in the 40th minute, only moments after Sami Khedira had headed wide from point-blank range.
Manuel Neuer was having a pretty quiet night in the Germany goal, but he showed his alertness to tip Gil Vermouth's shot over the crossbar in the 55th minute.
That attempt was the spark to a more determined Germany with Mesut Ozil dragging a shot wide of the far post before Lukas Podolski missed two chances, first shooting over the bar and then straight at Ariel Harosh.
Thomas Muller missed Germany's next big chance as he pulled a shot wide, much to the increasing frustration of the 43,000 fans inside the Leipzig Arena.
They were given something to cheer in the 82nd minute when substitute Schurrle took matters into his own hands and, from 20 yards out, curled a shot into the top corner
Klose had a header tipped over in the 87th minute as Germany tried but failed to establish a greater margin of victory in the pouring rain.
It was nevertheless Germany's first win in 2012 after defeats to France and Switzerland and a step in the right direction ahead of Euro 2012.