After a 2-0 defeat in the first leg in Asuncion, the Brazilian side, coached by Cuca, managed to wipe out their deficit inside 90 minutes thanks to goals from Jo and Leonardo Silva.
Following a goalless extra-time period they then edged the shoot-out 4-3, prompting scenes of jubilation in the camp, with former AC Milan and Barcelona playmaker Ronaldinho telling Globoesporte.com: "I came back to Brazil for this.
"A few months ago, they said Jo and I were over, but we have proved the players are not always wrong."
Herminio Miranda and Matias Gimenez missed in the shoot-out.
Olimpia goalkeeper Martin Silva started the match smartly, keeping out a long-distance shot by Ronaldinho.
The visitors had chances to stretch their aggregate lead by Atletico Mineiro goalkeeper Victor came out on top in one-on-one duels with Fredy Bareiro and Alejandro Silva.
The opening goal came in the second minute of the second half as Ronaldinho crossed and Wilson Pittoni failed to clear properly, allowing Jo to beat Martin Silva with a right-footed strike.
It was Jo's seventh goal in this year's Copa Libertadores, making him the tournament's top scorer.
Atletico Mineiro had a second goal in their sights and only the crossbar denied Leonardo Silva in the 59th minute when his header threatened.
Another header from the same player was kept out by Silva following a Ronaldinho free-kick, and Olimpia's hopes were hit when defender Julio Manzur was shown a second yellow card and then red with six minutes remaining.
Two minutes later the goal that levelled the tie arrived as Bernard crossed from the right and Leonardo Silva, again a menace in the air, rose highest of all players and nodded in.
Olimpia resisted pressure from the home side in the extra 30 minutes, with Rever going closest when he headed Bernard's corner against the bar.
In the shoot-out, Atletico Mineiro's Alecsandro, Guilherme, Jo and Leonardo Silva all converted, while goalkeeper Victor saved Miranda's opening attempt that put Olimpia in trouble.
The title was decided when Gimenez hit the post from Olimpia's fifth attempt as Atletico Mineiro earned a place in the FIFA Club World Cup, to be played in Morocco in December.
Ronaldinho said: "Congratulations to Olimpia because reaching the final is not easy at all. That team deserves everything."
Coach Cuca told Fox Sports: "This club has more than eight million supporters and had never won an important title before. This is a huge triumph for this club."
Olimpia boss Ever Almeida was unable to hide disappointment and said: "I'm still proud of my players, of their effort.
"However, we missed a lot of goal chances and you cannot fail in the final. Our fans have to be proud of these players.
"I don't think Atletico Mineiro did more than us in the two legs, but they are the champions and we failed."