You're talking about the window closing, not the Finder crashing, right? The Finder is ALWAYS running.
If you're showing only the USB stick's contents in a window, and you eject that stick, the window will close. This is normal and has been for at least 25 years (for floppies back then, of course).
If you're looking at the USB stick's contents in a window, navigate to the topmost level of the stick, WITH NOTHING SELECTED, and close the window. That should get the window settings to stick.