(Writing this down so I can google it when I inevitably forget about it in the future)

Powershell supports Unicode in a custom prompt, but if it’s coming out garbled, then you probably forgot to save your profile.ps1 as UTF-8 WITH BOM.

This is UTF-8 (no BOM):

This is is UTF-8 w/ BOM:

Nice.

(BTW, the code for this custom prompt is here)