Prized harbourfront land in Sydney’s inner west will be opened up to developers as the Baird government pursues the state's largest urban renewal program since the Olympics.

About 80 hectares of public land around Balmain, Rozelle, Annandale, Lilyfield and Pyrmont – an area four times the size of Barangaroo - will be turned over to housing, retail, tourism, commercial, recreation and maritime use.

It includes residential apartments, a potential overhaul of the rundown Sydney Fish Market and the transformation of the decaying, disused White Bay power station into a "cultural landmark".

The government will hold an “international summit” in November, coinciding with the G20 in Brisbane, to hear from experts on the best way to transform what it calls an “urban wasteland”.

The future of the prime stretch of harbour curling around the Anzac Bridge, two kilometres from central Sydney, has been highly contested as developers and commercial interests compete for space with maritime activities and public use.

The land, known as the "bays precinct", contains some of the last vestiges of the harbour's maritime and industrial past – history some fear will be wiped out by upmarket development as the government seeks to maximise return on the land.

Premier Mike Baird said its redevelopment, which would span 30 years, "presents possibly the most exciting and ambitious urban renewal and city building opportunity in the world today", describing the land as "underutilised and degraded”.

He said the summit would “invite the greatest minds in the field of city building and urban regeneration … we owe it to future generations to make sure we get the renewal of such a significant area right”.

The government has identified land for reuse around Blackwattle Bay and Wentworth Park, which could include a redevelopment of the Fish Market, and a "revitalised fresh food marketplace".

Around Rozelle Bay and the Rozelle rail yards it has flagged a residential village, and relocation of maritime activities.

White Bay would be devoted to port use and its historic, disused power station could become a "cultural landmark". Previous suggested uses for the power station include film studios, a university campus, an art gallery or a museum.

The project will be led by UrbanGrowth NSW. The public and other groups will be invited to provide feedback.

Charter boats, industrial barges and boat repairers cluster on one side of the Anzac Bridge at Rozelle and Blackwattle bays. On the other, Glebe Island and White Bay offer the last remaining deep-water berths in Sydney Harbour that could be used as a port.

Glebe Island handles bulk dry goods and houses a temporary events centre.

Urban Taskforce chief executive Chris Johnson last year described Glebe Island and White Bay as "lazy assets". He envisaged a future residential and commercial zone with towers up to 60 storeys high.

But City of Sydney planning director Graham Jahn warned at the time that such development could preclude the bays' ability to "deal with the future", adding “in terms of city planning, it's not just about finding every single opportunity to turn over land to the most profitable outcome”.

He said the bays are a rare commodity in a harbour colonised by "waterfront housing for executives" and "a massive city like this, with the world's greatest natural harbour, must have at least a tiny portion dedicated to shipping and port use".

Planning Minister Pru Goward said the urban renewal program “represents a game-changer for Sydney, with the potential to create the next big destination after the iconic Sydney Opera House".