Your clothes clothes cannot time travel with you because they are not made of biological matter.