Adkins' future at the St Mary's helm has come under scrutiny following a difficult start to the season, with Saints languishing in the relegation zone ahead of Saturday's match against fellow strugglers Queens Park Rangers.
However, he has previously guided the club to back-to-back promotions and Cork is confident he remains the right man for the job.
"To be fair, the gaffer was having to face speculation about his future after just a few weeks of the season because results hadn't been that great," Cork said in the Daily Star.
"But to suggest things like that is just silly in my opinion. He's done a fantastic, remarkable job at Southampton and just doesn't become a bad manager in such a short space of time.
"It's a shame. He's done so well here and the fixtures haven't been kind either. It's far too soon to judge him because the Premier League is so much harder than the Championship.
"But he remains very positive. He must be under pressure, of course he is, but I've never seen him lose it. Even when things are bad he passes that positivity on to the players.
"I just don't see the point in changing the manager. He needs time to sort things out."
Cork is enjoying life at Southampton after spending time with a number of different clubs over the past few seasons, although an ankle injury at the start of the season has proved frustrating.
"Nigel Adkins was one of the main reasons why I decided to join Southampton," he said. "He was good to me at Scunthorpe and, like myself, coming down here was a big move for him to make.
"It has been a difficult five or six seasons for me, moving to clubs on loan, so it's great to settle here and last season was so enjoyable.
"It was tough, but promotion was well deserved. It's been frustrating being stuck on the sidelines, not being able to help, and I can't wait to get going.
"We all know how hard the Premier League is, but there's a strong feeling in the squad that we can pick up points with the players we have got."