Appleton was touted as a possible candidate for the Turf Moor vacancy and, although no official approach was ever made by the Championship club, he admits the speculation was proving a distraction.
But with several other managerial positions yet to be filled, including Crystal Palace, the rumours are unlikely to halt just yet.
"I'm hearing these Palace rumours now. It was Burnley before that and thankfully that has been filled now," Appleton told the Portsmouth News.
"That couldn't come quick enough for me is all I can say. It was a case of 'please just give Sean Dyche the job and let me get on with mine'.
"I've had it (speculation) for two-and-a-half weeks now. That's long enough."
