Fishermen say now they're staring at industry collapse