Emmanuel Ledesma fired Boro to a hard-earned 1-0 win hot on the heels of the 2-0 victory at Millwall.
It was the first time Boro have won back-to-back games in 2013 and the boss admitted self-belief made the difference.
"In my first few games the team played well but results did not go well and we made late mistakes too many times and that affects confidence," he said.
"But winning at Millwall has made a massive difference. The players were better in every area.
"When you are more confident you are more sure in your decisions. You try more, you succeed more and that grows through the team.
"Now we have beaten the team who are top of the table and that will make more confidence."
But Karanka admitted that his short spell at Boro and a string of late sucker punches had left him nervous in the closing stages.
"It was painful for me," he said,
"We could have scored two or three in the first 25 minutes but we didn't and so, with what has happened before, I was nervous until the very last second."
Burnley boss Sean Dyche said that it was a sign of Burnley's changed status that Boro celebrated the goal so intensely.
"Their bench's reaction told a story," he said.
"They were up and screaming; they were buzzing. You'd have thought they had won the lottery rather than scored against Burnley but that is a sign of how far we have come.
"In this league it is all about fine margins and in this game the margins were incredibly tight," he said.
"It is the little things that go against you. I thought there may have been a foul in the build-up to their goal when their man crashed into one of ours and they broke from the loose ball.
"I'm not going to complain because he is very experienced referee but I thought that had to be a foul.
"But we responded to that and grew into the game. We had chances, not gilt edged ones but Shay Given had to make a couple of good saves."
Boro's pressure told after 24 minutes as the livewire Ledesma cut in from the right to drill in a low 25-yard angled effort that skidded towards the near post and bounced over diving Burnley keeper Tom Heaton.
The visitors fought hard to take a point from the game but their efforts came to nothing and Boro held on to take the win.