The Spireites go into Saturday's local derby against Rotherham, who currently occupy the final play-off spot, six points behind the Millers.
Second-placed Gillingham visit the Proact Stadium next weekend before Cook's men travel to Cheltenham, third in the table, at the start of March.
Chesterfield have a home game against struggling Aldershot and a trip to Dagenham in between the Gillingham and Cheltenham matches.
Cook said: "The next two, three, four or five games are massive for us. We have a massive month ahead.
"We need to start winning matches and quickly. We need to start an unbeaten run and quickly. If we don't then the teams above us are picking up points and going away from us.
"We need to start bridging that gap. If we don't do it quickly then the gap will become too big to bridge and the season will peter out for us, which I obviously don't want to happen.
"We go there (Rotherham) on Saturday on the back of a win against AFC Wimbledon and we will be looking for another win."
