On Windows: You can press Windows key + R, then type %USERPROFILE%\.sfdx.
On Linux and Mac OS: ~/.sfdx.
In that directory you’ll find a json file for each login saved and named after your username in that org.
You can delete the ones you don’t need.