Atletico Madrid considered bringing Fernando Torres back to the club over the summer, but made no formal approach.
Stefan Johansen is feeling settled at Celtic and has no intention of pushing for a move any time soon.
Hulse is well down the pecking order at Loftus Road and has not featured for them this season after being left out of their 25-man Premier League squad.
Hulse scored three goals in 15 appearances for Charlton in a three-month loan spell at Charlton earlier this season.
Millwall have been in the market for forwards after seeing Darius Henderson leave for Nottingham Forest and Chris Wood head to Leicester.