Miranda headed Atletico ahead after David Villa had missed a 50th-minute penalty before Costa slotted a last-gasp spot-kick to put Los Rojiblancos six points clear of capital city rivals Real Madrid and seven ahead of Barcelona.
Cristian Sapunaru, who fouled Villa for the first penalty, was shown a red card in the 90th minute for bringing down Costa, who made sure of the result which leaves his side in pole position for the title.
Elche, meanwhile, remain in 15th place following the defeat, four points clear of the relegation zone with four games left to play.
Miranda's 72nd-minute header calmed the home side's nerves following a first half with limited chances and Villa's earlier miss.
Costa went close with a third-minute effort before Koke saw a 20-yard effort blocked by Elche defender Alberto Botia in the eighth.
Atletico goalkeeper Thibaut Courtois showed just why he is such hot property when he backed up a comfortable take from Javi Marquez with another in the 25th minute to stop the Elche midfielder, before keeping out Sapunaru's header with a fantastic save.
Atleti then had a golden chance to take the lead from the penalty spot in the 50th minute when Sapunaru brought down Raul Garcia in the box.
Villa stepped up to take the spot-kick and directed it towards the bottom right, and could only watch as Elche keeper Manu guessed the right way and made the save.
The clash started to get a bit feisty, with goal threat Costa bundled to the ground by two strong challenges in succession only to receive a yellow card himself due to his reaction to Alberto Rivera's foul.
The Spain striker tried his luck with a weak effort as Atletico struggled to break down the visitors' defence, but the breakthrough finally came for Diego Simeone's side in the 72nd minute.
Atletico winger Jose Sosa delivered an outswinging corner to the far post, where Miranda rose highest and headed a powerful ball into the top right hand corner to give the league leaders a valuable goal.
Costa had the chance to seal the win in the 88th minute when he burst into the box, but Manu raced off his line to thwart the striker.
However the Atletico man was not to be denied from the penalty spot after he was brought down by Sapunaru, and converted from the spot in the 90th minute.