Rivals Manchester United look set to reclaim the Premier League crown and there has been little Richards could do about it, only returning from long-term injury in Wednesday's win over Wigan.
"It has definitely been frustrating, especially when you see United top of the league," said Richards.
"After last season I thought we would have kicked on and maybe been a bit better, maybe retained it. We haven't done that but I think next season we will regroup and hopefully we will get it back."
Richards was injured against Swansea in October and has seen Pablo Zabaleta thrive at right-back in his absence, and the England international acknowledges it will be tough to win back his place.
He added: "Five-and-a-half, six months - it is quite frustrating to be out so long. When I came back I was playing really well. Now Zaba has been class this season.
"It is going to be hard to get back in but you know what football is like, one minute you are in and the next you are out. When I am in I am going to have to take my chance."
Despite a successful return against the Latics, the 24-year-old realises he is likely to be back on the bench for the trip to Tottenham on Sunday.
He said: "I think Zaba is going to be back in after the way he has been playing but if I am called upon I will give 100 per cent.
"Last time after injury I almost tried to rush back. This time I don't want to rush back. I want to take it steady and when I am called upon try to do a job.
"For me it is just getting back, getting some minutes under my belt, just to play 80 minutes was enough for me for now.
"I will just keep training, working hard, trying to get fitter and stronger. Hopefully I will get some more games before the end of the season."