Poyet's side entertain the Gunners in a fourth round tie on Saturday, with Arsenal heading to the AMEX Stadium having beaten West Ham 5-1 in the Premier League.
Brighton first-team coach Charlie Oatway was dispatched to the Emirates on Wednesday and the report he brought back gave Poyet cause for concern.
"Charlie went, and he's quite scared. Apparently they were unbelievable but I'm not surprised," said Poyet.
"I know the quality they have got and if they have a good day it will be practically impossible. So we need to make sure we do something special ourselves.
"We need luck and we need to be at our very best in every department. We need to put in the best tactical performance.
"We lost heavily to West Ham last year, and West Ham lost heavily on Wednesday so if you put one plus one it's easy to see the result.
"But it's a football game, things happen, there can be accidents, a sending-off, a decision from the referee. You need to wait for that."
Poyet played for Tottenham and Chelsea during his playing career but he remains a fan of Arsenal's style of play.
"I don't support Arsenal but I like to watch them, and that says a lot," he added.
"Arsenal are one of the teams I try to learn from. I understand the supporters want to win - I am the same - but there are different ways of winning. You can win one year and then go five years in the bottom half.
"I really admire Arsenal, it's a very good place and the manager who goes in there after Arsene Wenger will be very lucky."