Caborn was sports minister at the time the design of the Stadium was agreed but he campaigned unsuccessfully for it to be built with football in mind for the future, in an attempt to prevent the public purse from incurring extra costs.
The stadium will now be converted at a cost of between £150million and £190million, while West Ham will pay £15million for a 99-year lease on the ground, and are expected to commence their tenancy in August 2016.
Caborn said: "This is the biggest mistake of the Olympics and lessons should be learned from this.
"West Ham are basically getting a stadium costing more than £600m for just £15m and a small amount in annual rent (£2m).
"I do welcome the fact that the future of the stadium has finally been secured, but we should also realise that the public sector is picking up the tab.
"The mistake was made in 2006/7 when they (the Olympic Board) ruled football out of a retro-fit design, as was done successfully in Manchester with the Commonwealth Games stadium.
"I suggested retractable seating like the Stade de France in Paris but they insisted it should be a 25,000-seat athletics stadium.
"Time and again mistakes are made with Olympic Stadiums and the lessons should be learned for any future similar projects."