Former Derby striker Dickinson settled the tie from seven yards after striker partner Nicky Forster saw his shot blocked.
Dickinson capitalised on a goalkeeping error from Nathan Abbey in the third minute to fire Brighton ahead before Lee Tomlin levelled with a low shot 11 minutes later.
Veteran Brighton striker Forster restored the Seagulls' lead from the penalty spot after being brought down by Curtis Osano after 21 minutes, but striker Aaron O'Connor made it 2-2 five minutes before the break with a low shot into the corner.
Forster drilled a shot wide in a second half of few clear-cut chances, and the match looked to be heading towards a replay until Dickinson grabbed his sixth goal of the season.
Rushden, who lost Neil Cousins and substitute Matt Pattison through injury in the first half, put up a courageous display throughout as they slumped to their first defeat in eight games.