feat: Autoswap improvements (skip unload per model type)

[esolithe]

What's new

• Adds options to skip swapping for certain model types during autoswap, such as embedding models which may be lightweight enough to keep in memory as other stuff runs

[LostRuins]

This seems like a very janky approach. I think we can do better by making the autoswap for small models intelligent - if the model file is less than say 1gb, disqualify it from autoswap.

[henk717]

I prefer both approaches combined. This one gives the user the desirable flexibility while your idea is a better default.

[LostRuins]

too many single purpose flags for me tho

[esolithe]

This seems like a very janky approach. I think we can do better by making the autoswap for small models intelligent - if the model file is less than say 1gb, disqualify it from autoswap.

Frankly this is more limiting to the user in my mind? Not everyone has that extra GB. Or some might have plenty more. Trying to guess every combination isn't something I would expect to automatically work well.

The flags are in a python dict so it isn't separated out within the code itself - and for a launcher those many options don't change much because a use is just ticking boxes. But if you wanted to make the argument for it singular maybe that would work? Like a list of the excluded types which translates to Boolean flags in the config / code?

[henk717]

One command where you specify what to pin sounds logical to me. Prevents the clutter and keeps the customizability.