Rovers are in serious danger of suffering back-to-back relegations, leading to further unease among some fans over Venky's ownership of the club.
But Singh has hit back, questioning where investors were when the club was previously up for sale.
"There is always a lot of talk when things are not going very well. Every Tom, Dick and Harry thinks they can own the club, but don't forget that my bosses (Venky's) have been only there for around two years," he told Goal.com.
"Where were these people when the club was for sale for many years? Hence, sometimes you have to take it with a pinch of salt.
"There are people who would not be able to run a bar and they think they can run a football club, so I wouldn't pay any attention or waste my time on them."
Singh also promised there will be a "cleansing" at the club as they attempt to improve their fortunes next season.
"We find ourselves in a situation that we are fighting to stay in the Championship and I would not like to say anything to disrupt that," added Singh.
"But you know we have identified the reasons and identified the remedy to ensure things go smoothly in the future. Clubs have challenges and these can appear anywhere.
"To be frank and almost brutal, there needs to be a cleansing of sorts at the club. I am not at a liberty to say so, but over a period of time you'll understand.
"The fans are very important to us and we hope to attract more fans. I have built a good relationship with the fans over there, and they understand the job needs everyone's support and have been very supportive."