Villa are three points clear of 18th-placed Wigan, but Roberto Martinez's men have two games in hand, the first of which is at Manchester City on Wednesday night.
"The main thing we have to think about is ourselves," said Baker. "It's in our hands and that's our focus, what we do.
"It's about us and what we do for the rest of the season. If we carry on playing like we have been we're confident we'll be okay.
"We're hoping to be safe by the time we go to Wigan (on the final day of the season), that's what everyone wants to do.
"But if we had to go there and get a result we'd just keep going and doing what we've been doing. We're confident and we know we can do it.
"We just need to keep taking the points. The point against Fulham was another step towards the finishing line.
"We've picked up some good results lately with ten points from five games, which is a really good return."