The League One club want a groundshare at the stadium, which is close to their current Brisbane Road home, and claimed the London Legacy Development Corporation was wrong to award sole tenancy to the Premier League Hammers.
Lawyers representing Orient asked a High Court judge to give the club the go-ahead to pursue a judicial review of the Corporation's decision.
But Mr Justice Lewis, who also heard arguments from lawyers representing the LLDC and West Ham, refused their request, leaving Orient to consider their next course of action .
In a statement on their official website, Orient said: "We feel a real sense of injustice at today's decision.
"(West Ham vice-chairman) Karren Brady told the House of Lords only a few weeks ago that West Ham United had no objection to a ground share with Leyton Orient, but West Ham United's barrister today claimed that to groundshare would adversely affect the club because the Olympic Stadium is "part of the brand".
"Our real concern is the lack of transparency that has been shown throughout the process by a public body.
"It is deeply disappointing that both the Court and the LLDC have made decisions based only on financial considerations, when the purpose of the Stadium's legacy was regeneration of the area with a community focus.
"We believe that the LLDC exercised its discretion to favour West Ham United, no doubt under pressure from West Ham United to make them sole football tenants for the benefit of their "brand".
"Delivering a new brand to West Ham United was not the intended purpose of the Olympic Stadium, and we now have to look to the House of Lords to find a common sense solution for Olympic legacy and local community.
"As the judge noted, Leyton Orient are flying high in League One. We will enjoy our current on-pitch success whilst we contemplate how to proceed in the best interests of the club."
West Ham were named as the number one choice to move into the stadium in December, but Orient chairman Barry Hearn has consistently argued the process was flawed.
Nevertheless the Hammers are expected to move from Upton Park into their new ground from August 2016.
The club also issued a statement on their website, welcoming the news.
It read: "West Ham United welcome the decision to not grant permission for a judicial review into the LLDC's process that awarded the club the opportunity to make the Olympic Stadium its home in 2016.
"Although the application for a judicial review would not have had any impact on West Ham United's move to the Queen Elizabeth Olympic Park, the club have always believed the process was robust, fair and transparent.
"The club and other key stakeholders in the Queen Elizabeth Olympic Park can now focus fully on progressing their ground-breaking plans to create a stunning venue of which the whole country can be proud, alongside a long-lasting and tangible Olympic legacy with a real community club at its core.
"West Ham United will continue its consultation with supporters to ensure the Hammers' new home is the pride of east London and the envy of the rest of the footballing world."