Antoine Griezmann scored in the 82nd minute to give France a deserved lead after they had dominated the match in both chances and possession, but Caceres, who has played more than 50 games for his country without finding the net, headed the equaliser in the last minute.
It meant the night would end in frustration for France manager Didier Deschamps as his side lacked a clinical finish up front and fell asleep at the back as Paraguay showed they had a sting in their tail.
It had all looked so promising for the 1998 World Cup winners, with strikers Loic Remy and Oliver Giroud linking up well and putting the Paraguay defence under all sorts of pressure until the former was replaced midway through the second half.
Remy, who has been linked with a move to Giroud's club Arsenal, was a handful in the first half and showed his intent early on when he turned well near the edge of the box and just fired wide of the near post.
Giroud then almost freed the QPR man 10 minutes later when he raced into the box and headed the ball to Remy at the far post, but the striker was just unable to control it under pressure and sent the ball over the bar. Giroud pulled the trigger himself shortly afterwards but Luis Cardozo made the block.
France midfielder Paul Pogba then ran across the edge of the 18-yard box but fired wide.
Remy went agonisingly close to scoring an amazing scissor-kick goal after a great run and skill on the right from Giroud, who got a cross away under pressure.
Blaise Matuidi then sidefooted across the face of goal as France were unable to make the breakthrough before the break.
Paraguay went close to scoring from a free-kick early in the second half but Les Bleus goalkeeper Hugo Lloris did enough to keep it out after France's two-man wall separated.
Giroud was almost on target with a looping header shortly after which almost dipped under the bar, but Paraguay keeper Anthony Silva kept it out and again denied the France striker from the resulting corner.
It was becoming an increasingly frustrating game for Deschamps' men, with the game still deadlocked at 0-0 by the hour mark.
Remy was replaced in the 64th minute by Griezmann while Clement Grenier also came on from the bench for Matuidi as the France coach looked to mix things up.
Grenier's free-kick in the 80th minute was dealt with by the Paraguay defence with ease before Moussa Sissoko raced down the right and put the visitors under real pressure.
A goal always looked to be on the cards and it finally came with eight minutes remaining. Silva again denied the dangerous Giroud but the ball worked its way out to Griezmann, who struck it sweetly from the left of the area into the far corner.
It looked like a Paraguay defender may have touched the ball before it went into the net, but the goal was awarded to the Real Sociedad winger.
Griezmann then tried to return the favour to Giroud with a burst down the left and a quick ball in, but Silva raced forward to claim it and neutralise the attack.
However, Paraguay had the last word as the match finally sparked into life, when a free-kick from a central position caught France flat and Caceres opened his account.