Brendan Rodgers' side just missed out on the Premier League title last season despite scoring 101 goals, as they conceded 50 times.
Liverpool again looked vulnerable at the back in Sunday’s 2-1 victory over Southampton, but Skrtel believes it is unfair to say that only the defenders are at fault.
The centre-back said: "Obviously it hurt but it’s not only about the back four, it's about the team and defending as a group.
"Because we conceded 50 goals, some people are blaming the back four, but it isn’t just about the back line, it is about defending as a team and that’s the big thing that we need to improve.
“If we do that we will stop people from talking about the defence.
"If we can do that and also cut out some of the individual mistakes it will make a big difference.”
Skrtel believes it is a matter of striking the right balance between attack and defence and is confident Liverpool will address the problem.
“We are a team that likes to attack and a lot of players like to get forward so that is maybe something that we need to work on because not everyone can go forward, sometimes you have to defend too,” he added.
“We’ll see anyway, we will work on it and try to improve it. As players we are not happy with that record but we know the direction we were going in as a team last season and we will try to improve on that in the new one.”
Rodgers has strengthened his defence already this summer with the capture of Dejan Lovren from Southampton, and Skrtel thinks that can only help Liverpool’s chances.
He explained: “There’s a lot of competition for places but that can only be good for us as players because when you know that someone is waiting to play in your position it pushes you to show the manager how good you are.
"That can only be good for the team and it’s up to the manager who starts.”