A gradio Chatbot with Gemma3-1b-it GGUF
A full locally running chat-bot powered by llama.cpp server and Gemma3-1b-it-GGUF
- Clone the repo
- create a
venv(optional) and install the dependencies
# create a virtual environment
python -m venv venv
# activate the virtual environment
venv\Scripts\activate
# install the dependencies
pip install openai tiktoken gradio
- download the GGUF file in the same directory from the Bartowski repository google_gemma-3-1b-it-Q8_0.gguf
from the terminal run
python gr_Gemma3.py