Vorm was left frustrated after seeing his side lose 2-1 to West Brom last weekend, feeling they did not cope with the opposition's physical style of play.
Dutch ace Vorm believes they need to add some steel to their free-flowing style of football if they are continue their progress under Michael Laudrup.
"It's about attitude and mentality," Vorm told the Swansea Evening Post.
"You have to not be afraid to put your foot down because the opposition do that.
"It's always us who are on the receiving end of elbows and kicks. We can also do it.
"It's part of us maturing our game, because teams can't stop us football-wise when we play so well even on a bad pitch.
"The chances we created (at West Brom) were a joy to watch. We didn't have to lose the game and that annoys me a bit.
"Most of the other teams are physically stronger than us, but sometimes we have to show that we are not only a team who plays good football.
"We also have to show the strength and power you need. The back four is not a problem, but the whole team need to do it.
"We have to add a physical element. You could see that we were better than the opposition football-wise (at West Brom).