Saturday's 1-0 win over Charlton was the Teessiders' fifth in six league games and left them sitting ninth in the table, five points off the play-off places.
The Foxes, who lead second-placed QPR by a similar margin, provide the opposition at the King Power Stadium this weekend, and Given knows it will take a much-improved display to return with something to show for their efforts.
He told Boro TV: "We didn't play well and the manager and a good portion of the fans will probably agree with me.
"But it's important we grind the results out. We kept a clean sheet, we sneaked a goal so job done, and those three points are massive.
"We are up to ninth and looking up the table, but the next game is always the most important one - and the next one is Leicester away and we know how difficult that will be."