Jordan Rhodes scored his 12th of the season with a fantastic finish after 22 minutes but Martyn Waghorn equalised after a neat team move just after half-time.
Marshall arrived in the box unmarked to head Rovers back in front after 48 minutes and David Dunn's first goal since August five minutes from time gave them a cushion.
They conceded late on when Steve Morison bundled in after a Simon Eastwood howler but held on for a second successive home win.
They have now lost just once in their last nine at home and Bowyer feels Marshall is emerging into an important player.
"He showed in the build-up to the QPR game what he's capable of when he gets his head down and I think the performance against QPR gave him loads of confidence," said Bowyer.
"It's a fantastic header from a great Tommy Spurr cross. That should fill him with confidence even more.
"We know he has talent and know he wants to succeed."
The game was the first time Blackburn have scored more than one goal in a single game at Ewood since the end of August and though Bowyer conceded his side weren't at their best, he was happy to see their character.
He said: "We scored three good goals. We didn't play particularly great by our standards but we've played a lot better and lost games.
"We're happy with the character and resilience we've shown against a tough Millwall team.
"I didn't think we passed the ball well enough today by our standards. Sometimes the conditions were not great but you have to give credit to the opposition.
"Millwall got amongst us but I was pleased that we stuck at it and showed some fighting spirit again. It's about time we won a game when not at our best."
Millwall manager Steve Lomas questioned the desire of his defenders after yet another poor defensive performance undid much of their good attacking work.
Their away record makes unpleasant reading, with only Barnsley picking up less points on the road, and Millwall have conceded more goals than any in the division.
They created chances and Waghorn's strike came at the end of a silky team move, but after watching his team lose their fifth game in the last six away Lomas was understandably upset.
He said: "We were the masters of our own downfall.
"The three goals we conceded were, I wouldn't even say schoolboy - probably under-12s.
"And that's the reason we're not looking up because going forward we're okay, a threat, scoring goals, but all season we've given away cheap, cheap goals.
"And again it's raised its ugly head. We've scored two goals away from home and should be coming home with at least a point.
"I can't keep saying it's unlucky. We're going to have to do something about it.
"It's about having the desire in the box. If you don't get a head on it, making sure the guy you're marking doesn't get a free header or at the very least he's heading your head and he's got a nasty cut on his face or breaks a nose.
"I don't see too many forwards break a nose against us.
"They've all been given rules and responsibilities. We work on it but ultimately when you're out there, you have to deal with it yourself. It's a recurring theme.
"Ultimately it's about individuals having desire and it's just not good enough at this level."