You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What model variant did you use and what GPU did you use?
One guess is that you may run out of GPU memory if you try to run the 7B un-quantized model on a 16GB GPU. You can either try the 7B quantized model or a 2B model and it should work.
When I run
It returns the error:
docker: Error response from daemon: could not select device drit device driver "" with capabilities: [[gpu]].
while if I run on CPU with command:
It works out OK.
The text was updated successfully, but these errors were encountered: