Alan Pardew's troops are not in action this weekend as the FA Cup quarter-finals take place.
Newcastle should have been facing Everton at St James' Park, but the Toffees are taking on Arsenal, so United's next game is a trip to Fulham on Saturday, March 15.
Their superb 4-1 win over Hull City, which was marred by Pardew's dismissal to the stands for an altercation with David Meyler, moved them onto 43 points from 28 games - and they are 12 points better off than 10th-placed West Ham.
With that in mind, Williamson has urged his team-mates to look up the table, with sixth-placed Everton just five points ahead of them, as they bid to ensure the season doesn't fizzle out.
The defender told the Newcastle Chronicle: "Everyone has got something to play for. If we can get a couple of good results there are targets. Looking up in the top half and the Europa League, that's what we’ve got to push for.
"We need to make sure we get the consistency we've had in the season."
Williamson insists the players are focused purely on what happens on the field, adding: "Off the pitch there might be things going on but that doesn't affect us. We know our jobs and what we get paid for.
“We know where we want to get to. That focus will keep us going.”
On emerging as a first-choice centre-back alongside Fabricio Coloccini, Williamson said: "It's just hard work, persistence and belief.
"There's no secret. Every player gets frustrated when they are not playing because that's what you are here to do.
"That's where your passion belongs. Everyone has aspirations to play no different to myself.
"It was about keeping that belief and knowing when I got my chance it would happen.
"You've got to be fit and strong enough to take that. Thankfully that's what happened."