The Welsh club shipped three goals last week as they crashed to a 3-1 loss at the hands of Northampton.
But they responded in perfect fashion as they took the spoils in the first competitive derby with neighbours Rovers in 25 years thanks to Andy Sandell's first-half penalty.
Edinburgh said: "It's all the more sweeter being in a derby. When you see the atmosphere and the amount of numbers here, you want to reward the fans with a victory and performance.
"It was a hard-fought battle, a well earned three points in a real fiercely contested derby.
"Both teams really went for it but I'm delighted we bounced back after such a hard defeat to take last week. After conceding three poor goals last week we kept a clean sheet for the first time.
"To keep a clean sheet when our young defence has just been thrown together was really pleasing."
Edinburgh added: "We had good chances to possibly win by more but we're creating chances.
"We're more than delighted with our start. We don't like losing, as a group we're not used to it."