When an alien enemy is required by the President, or by order of any court, judge, or justice, to depart and to be removed, it shall be the duty of the marshal of the district in which he shall be apprehended to provide therefor and to execute such order in person, or by his deputy or other discreet person to be employed by him, by causing a removal of such alien out of the territory of the United States; and for such removal the marshal shall have the warrant of the President, or of the court, judge, or justice ordering the same, as the case may be.