Fight to keep Victorian heritage lost