(Writing this down so I can google it when I inevitably forget about it in the future)
Powershell supports Unicode in a custom prompt, but if it’s coming out garbled, then you probably forgot to save your profile.ps1 as UTF-8 WITH BOM.
This is UTF-8 (no BOM):
This is is UTF-8 w/ BOM:
(BTW, the code for this custom prompt is here)