On the fourth anniversary of the first evacuation, owners are pushing for a scheme to clear their debt, similar to the one for Lismore flood victims.
Apartment owner Treacy Sheehan and her five-year-old son were forced to evacuate in 2019 due to the severe structural cracking.
"I open my bank account and it says minus one million... it's devastating," Sheehan said.
"It's been painful. (The) first year, we moved five times."
Sheehan said all of the initial paperwork showed there was no issue with the building.
Owner Jie Ding bought her million-dollar apartment just two months before she was ordered out.
"Whenever I pass by this building, I just feel so sad," Ding said.
The owners are currently getting up to $400 a night in rental relief but it runs out in June.
The NSW government has been advised buying the towers would cost $50 million.
It has already spent $9 million on resident accommodation.