Mr Andrews has reiterated he wants no part of the New Zealand travel bubble that started on Friday, suggesting 55 Kiwis had turned up in his state after landing in NSW.

"We have been able to find 23 ... we are still working to find the balance," he told reporters on Sunday.

"We have been given a list, 12 hours after they arrived. We are ringing them, one of them was in Byron Bay."

Victoria's health department later confirmed all travellers had been contacted while clarifying three hadn't even entered the state.

"Three of the travellers did not cross the border into Victoria and remain in NSW, and one who was in Victoria returned to NSW today," a statement said on Sunday.

Acting Immigration Minister Alan Tudge insists Victoria's chief health officer Brett Sutton was at the meeting where authorisation was given for individuals who arrived in Sydney from New Zealand to then travel to Victoria.

"So the Victorian government was present when it was discussed, they were made aware that this was going to occur, they raised no objections in the meetings," Mr Tudge told reporters.

But Mr Andrews said this was not the case.

"Seriously, my advice to minister Tudge is, instead of stubbornly defending this, work with us and let's make sure Victoria is not part of a bubble we never agreed to be in," the premier said.

Mr Andrews has been able to ease some of his state's stiff COVID-19 restrictions after reporting just two new cases on Sunday after one on Saturday.

Among the state's relaxation of restrictions, from Monday Melburnians will be allowed to travel 25km from home, while there will be no limits on time spent away from their residence.

But the state capital's businesses must wait longer for eased restrictions, with changes to retail, hospitality and "personal care" services pushed back until November.

"I know and understand that not everything that everybody wanted is in the announcements I have made today," Mr Andrews said.

"I just make this point, I have announced today what is safe."

In a joint statement, Prime Minister Scott Morrison, Treasurer Josh Frydenberg and federal Health Minister Greg Hunt urged a rethink on the timeline for businesses reopening.

"We understand and sympathise with the frustration and concern expressed by businesses about the ongoing cost and impact on their jobs and livelihoods of this second wave," the trio said.

"Every day Victoria remains under restrictions to get the second wave in Victoria under control comes at a heavy cost.

"Throughout the lockdown more than 1000 jobs have been lost, on average, every day."

Mr Andrews had previously accused Mr Hunt of "playing games" after he urged the Victorian leader to move to the next step in line with NSW.

Federal Labor frontbencher Tanya Plibersek was unimpressed.

"It's actually really gross that federal Liberal MPs are trying to use this as an attack on Daniel Andrews," she told ABC television's Insiders.

"He hasn't done it for popularity, he's done it because he's considered it the right thing to do.

"The federal government ought to be looking at its own role in Victoria's difficulties including aged care."

Australian Associated Press