If any merchandise or baggage is laden on, or unladen from, any vessel or vehicle without a special license or permit therefor issued by the appropriate customs officer, the master of such vessel or the person in charge of such vehicle and every other person who knowingly is concerned, or who aids therein, or in removing or otherwise securing such merchandise or baggage, shall each be liable to a penalty equal to the value of the merchandise or baggage so laden or unladen, and such merchandise or baggage shall be subject to forfeiture, and if the value thereof is $500 or more, the vessel or vehicle on or from which the same shall be laden or unladen shall be subject to forfeiture.