Ich versuche, ein Gradio-Backend zu starten, aber mir scheinen Bibliotheken zu fehlen, obwohl ich sie heruntergeladen habe.
(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ gradio gradio-chatbot.py
Warning: Cannot statically find a gradio demo called demo. Reload work may fail.
Watching: '/home/reply/.local/lib/python3.10/site-packages/gradio', '/home/reply/Documents/chatbot-rag', '/home/reply/Documents/chatbot-rag'
/home/reply/.local/lib/python3.10/site-packages/langchain/__init__.py:34: UserWarning: Importing PromptTemplate from langchain root module is no longer supported. Please use langchain.prompts.PromptTemplate instead.
warnings.warn(
Initializing backend for chatbot
/home/reply/.local/lib/python3.10/site-packages/transformers/pipelines/__init__.py:694: FutureWarning: The use_auth_token argument is deprecated and will be removed in v5 of Transformers.
warnings.warn(
Traceback (most recent call last):
File "/home/reply/Documents/chatbot-rag/gradio-chatbot.py", line 10, in <module>
backend.load_embeddings_and_llm_models()
File "/home/reply/Documents/chatbot-rag/backend.py", line 50, in load_embeddings_and_llm_models
self.llm = self.load_llm(self.params)
File "/home/reply/Documents/chatbot-rag/backend.py", line 66, in load_llm
pipe = pipeline("text-generation", model=self.llm_name_or_path, model_kwargs=model_kwargs)
File "/home/reply/.local/lib/python3.10/site-packages/transformers/pipelines/__init__.py", line 834, in pipeline
framework, model = infer_framework_load_model(
File "/home/reply/.local/lib/python3.10/site-packages/transformers/pipelines/base.py", line 269, in infer_framework_load_model
model = model_class.from_pretrained(model, **kwargs)
File "/home/reply/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 565, in from_pretrained
return model_class.from_pretrained(
File "/home/reply/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2614, in from_pretrained
raise ImportError(
ImportError: Using load_in_8bit=True requires Accelerate: pip install accelerate and the latest version of bitsandbytes pip install -i https://test.pypi.org/simple/ bitsandbytes or pip install bitsandbytes
Ich habe sie also heruntergeladen:
(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ python3 -m pip install accelerate
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: accelerate in /home/reply/.local/lib/python3.10/site-packages (0.24.1)
Requirement already satisfied: psutil in /home/reply/.local/lib/python3.10/site-packages (from accelerate) (5.9.6)
Requirement already satisfied: packaging>=20.0 in /home/reply/.local/lib/python3.10/site-packages (from accelerate) (23.2)
Requirement already satisfied: pyyaml in /usr/lib/python3/dist-packages (from accelerate) (5.4.1)
Requirement already satisfied: torch>=1.10.0 in /home/reply/.local/lib/python3.10/site-packages (from accelerate) (2.1.0)
Requirement already satisfied: numpy>=1.17 in /home/reply/.local/lib/python3.10/site-packages (from accelerate) (1.26.1)
Requirement already satisfied: huggingface-hub in /home/reply/.local/lib/python3.10/site-packages (from accelerate) (0.17.3)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.105)
Requirement already satisfied: filelock in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.13.1)
Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (8.9.2.26)
Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (10.3.2.106)
Requirement already satisfied: sympy in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (1.12)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.105)
Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (11.0.2.54)
Requirement already satisfied: networkx in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.2.1)
Requirement already satisfied: jinja2 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.1.2)
Requirement already satisfied: fsspec in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (2023.10.0)
Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (11.4.5.107)
Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.0.106)
Requirement already satisfied: triton==2.1.0 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (2.1.0)
Requirement already satisfied: nvidia-nccl-cu12==2.18.1 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (2.18.1)
Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.3.1)
Requirement already satisfied: typing-extensions in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (4.8.0)
Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.105)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /home/reply/.local/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.105)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /home/reply/.local/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.10.0->accelerate) (12.3.52)
Requirement already satisfied: tqdm>=4.42.1 in /home/reply/.local/lib/python3.10/site-packages (from huggingface-hub->accelerate) (4.66.1)
Requirement already satisfied: requests in /home/reply/.local/lib/python3.10/site-packages (from huggingface-hub->accelerate) (2.31.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/reply/.local/lib/python3.10/site-packages (from jinja2->torch>=1.10.0->accelerate) (2.1.3)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->huggingface-hub->accelerate) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in /home/reply/.local/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (2023.7.22)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/reply/.local/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (2.0.7)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/reply/.local/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (3.3.1)
Requirement already satisfied: mpmath>=0.19 in /home/reply/.local/lib/python3.10/site-packages (from sympy->torch>=1.10.0->accelerate) (1.3.0)
(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ python3 -m pip install -i https://test.pypi.org/simple/ bitsandbytes
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://test.pypi.org/simple/
Requirement already satisfied: bitsandbytes in /home/reply/.local/lib/python3.10/site-packages (0.41.1)
Ich denke, das Problem ist, dass der Python-Interpreter nicht in der virtuellen Umgebung ist:
(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ python --version
Command 'python' not found, did you mean:
command 'python3' from deb python3
command 'python' from deb python-is-python3
(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ python3 --version
Python 3.10.12
(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ which python3
/usr/bin/python3
Ist meine Diagnose richtig?
