All the goals came in the second half following a lacklustre performance by both teams before the interval but, when Rovers continued to struggle in the second period, the visitors raised their game and finished worthy winners.
Francois Zoko fired Stevenage in front following a mistake by Paul Quinn in the 49th minute and Peter Hartley headed home unchallenged following a 65th-minute corner.
Harry Forrester pulled one back in the 72nd minute with a spectacular 25-yard volley but Rovers were caught pushing too many players forward when Darius Charles added Stevenage's third in the last minute of normal time.
Teenage right-back Liam Wakefield hammered home a second for Doncaster deep into stoppage time but it came far too late to make a difference to the outcome.
Dickov did not mince his words after the match and said: "I'm not just disappointed with the defeat or the manner of it - I'm raging.
"It's the cup and obviously we want to do well.
"But we were devoid of ideas and everything. In fact we were totally inept and I was disgusted with the performance.
"It's all about maintaining high standards and we are not doing that so I'm going to have to sort it out.
"If it comes to getting one or two players out, or even five or six, then we are going to have to do that because they've all had a chance.
"We've had horrendous luck with injuries and this performance was a massive contrast to how we played against QPR on New Year's Day when we should have had something from the game.
"We had three quality players out today but there are no excuses whatsoever.
"I'll back by players to the hilt but there comes a time when you start to look stupid doing that.
"The performance simply underlined how much we need to strengthen the side."
Delighted Stevenage boss Graham Westley said the victory had not surprised him in the slightest.
He added: "We've been struggling with injuries and had to put a lot of youngsters in but when our best players are available we can compete - even with Championship sides.
"We probably benefited through not playing midweek which gave us the chance to have a good look at Doncaster against QPR.
"They were without Richie Wellens, Federico Macheda and Theo Robinson and that probably helped us because they are class players.
"But we played well today and I'm really pleased for the players.
"The FA Cup is of massive importance to Stevenage.
"It's really essential because it's the lifeblood for us and the further we go helps the club prosper.
"I hope it has a knock on effect in the league because we need to win ten games to ensure we keep our League One status."