It's allowed but the rule to remember is that any clothes that you cannot wear in front of general public/people, you should not pray in those clothes because you should atleast dress in front of Allah how you would dress infront of people.