Every permit issued in accordance with the provisions of sections 443 to 448 of this title, which may not be taken up by an inspector or deputy inspector, shall be returned within four days after issuance to the office of the supervisor of the harbor; such permit shall bear an indorsement by the master of the towboat, or the person acting in such capacity, stating whether the permit has been used, and, if so, the time and place of dumping. Any person violating the provisions of this section shall be liable to a fine of not more than $500 nor less than $100.