The Clarets had looked course to continue their 100% start to the season when Charlie Austin headed home the opener shortly before the break in the Championship clash on Teesside.
But Nicky Bailey equalised with a screamer from distance a minute later, before Adam Reach lashed in a thunderbolt late in the second half.
Junior Stanislas converted a header to seemingly limit the hosts to a solitary point but Williams stunned the travelling fans with yet another Boro rocket to grab all three on offer.
With defeat at Barnsley marking an inauspicious start to the campaign, there was a heightened sense of expectation back at The Riverside, where last season's poor displays put paid to any hopes of promotion.
There was no such anxiety for a buoyant Burnley side, though, with Eddie Howe's men fresh from a convincing opening-day victory over Lancashire rivals Bolton.
Fans of both sides arrived intrigued by Boro's surprise loan capture of Chelsea starlet Josh McEachran and he duly replaced veteran Julio Arca in midfield.
Youngsters Curtis Main and Reach also earned first league starts to give the team a youthful dynamism.
Indeed there were six home debuts, with George Friend, Grant Leadbitter and Emmanuel Ledesma also strutting their stuff on Teesside for the first time.
Burnley made one change, Howe resting one of Saturday's goalscorers, Martin Paterson, and fielding Sam Vokes.
McEachran soon showed why a host of Premier League clubs were interested in his services as he dusted a series of passes through a packed midfield before Boro had their first solid chance on goal.
In the 25th minute Chris McCann conceded a free-kick for a foul on Reach, Ledesma obliging to whip the ball just beyond the near post.
The hosts continued to grow in confidence and might have scored if Justin Hoyte had connected a toe to Ledesma's canny through-ball.
Reach then dragged a shot just wide of the target before Burnley took the lead in the 41st minute.
Pressing forward, Dean Marney fed Kieran Trippier who crossed cleanly from the right for Austin to steal in and head home.
The boisterous celebrations of the away fans were muted one minute later, however, as Bailey silenced the south stand with a stunning daisycutter.
After Main competed for a heavy ball forward with Clarets skipper Jason Shackell, Bailey looked up just once before rifling the rebound into the bottom corner from 20 yards.
Following five frenetic pre-interval minutes, the second half got off to a more sedate start.
It was not until the 56th minute that McEachran reinvigorated the home faithful by heading Reach's diagonal cross against the turf and wide.
Joseph Mills missed the target with a left-footed drive at the other end before Marvin Bartley replaced Vokes on the hour mark.
But Boro continued to pass the ball about incisively, and Main almost emulated Bailey as he drove a fierce shot against Lee Grant's outstretched glove in the 68th minute.
Reach very nearly headed the second in the 75th minute, Grant standing tall to deny the young winger. Soon Leadbitter picked his spot but pinged the ball wide as Boro pressed on.
A nervous Howe replaced Ross Wallace with Ben Mee, but it was to no avail as Reach picked up a loose ball to scoop a wonderful drive across Grant and into the back of the net just two minutes later.
Stanislas edged a goalmouth scramble to scuff a header beyond Jason Steele and seemingly stifle the celebrations.
But it was Williams, a tactical substitute for the tiring Main, who had the final word with an unstoppable drive into Grant's top corner.