Ball, a first-half substitute for injured new signing Nicky Ajose, secured three points for Posh after Grant McCann had cancelled out Sean Scannell's opener.
Palace were good value for their half-time lead but they caved in after the break.
The Eagles almost scored straight from the kick-off when Posh gave Scannell a sight of goal and Paul Jones was forced to tip the winger's shot around the post.
At the other end George Boyd, who has been linked with a move away from London Road this summer, went on a trademark jinking run before playing in Grant Basey whose cross was headed wide by Lee Frecklington.
Palace keeper Julian Speroni was called into action in the 12th minute, finger-tipping the ball over the crossbar after McCann let fly from 30 yards.
And the visitors took the lead in the 33rd minute when some hesitation in the Posh defence saw the ball ricochet to Scannell, who cleverly flicked it over the advancing Jones and into an empty net.
Tommy Rowe almost hauled Posh level moments later but his shot was well saved by Speroni.
Peter Ramage, one of three debutants in the Palace back four, saved his side before the break - although he did not know much about it - when he blocked a fierce drive from Ball on the goal-line.
Peterborough roared out of the blocks in the second half and Lee Tomlin headed Rowe's cross over before Boyd's long-range drive flew wide.
Palace striker Glenn Murray, making his debut, could consider himself fortunate to stay on the pitch when, having already been booked, he clattered Basey and escaped with a lecture.
Another Palace debutant, defender Andrew Davies, could have doubled his side's lead but he headed Darren Ambrose's cross over.
And instead Posh grabbed an equaliser in the 58th minute when Tomlin took a free-kick near the corner flag.
He ignored the crowd of players in the penalty area and instead played the ball into the path of McCann, who drilled a first-time shot low into the corner of the net from the edge of the box.
The winner came 17 minutes from time when Mark Little wriggled down the right wing and floated over a cross for Ball to nod in at the far post.