├── doc └── screenshot.png ├── docker-compose.yml ├── Dockerfile └── README.md /doc/screenshot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mattcurf/ollama-intel-gpu/HEAD/doc/screenshot.png -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | services: 2 | ollama-intel-gpu: 3 | build: 4 | context: . 5 | dockerfile: Dockerfile 6 | args: 7 | IPEXLLM_RELEASE_REPO: ipex-llm/ipex-llm 8 | IPEXLLM_RELEASE_VERSON: v2.2.0 9 | IPEXLLM_PORTABLE_ZIP_FILENAME: ollama-ipex-llm-2.2.0-ubuntu.tgz 10 | container_name: ollama-intel-gpu 11 | restart: always 12 | devices: 13 | - /dev/dri:/dev/dri 14 | volumes: 15 | - ollama-intel-gpu:/root/.ollama 16 | environment: 17 | - ONEAPI_DEVICE_SELECTOR=level_zero:0 18 | - IPEX_LLM_NUM_CTX=16384 19 | ollama-webui: 20 | image: ghcr.io/open-webui/open-webui 21 | container_name: ollama-webui 22 | volumes: 23 | - ollama-webui:/app/backend/data 24 | depends_on: 25 | - ollama-intel-gpu 26 | ports: 27 | - ${OLLAMA_WEBUI_PORT-3000}:8080 28 | environment: 29 | - OLLAMA_BASE_URL=http://ollama-intel-gpu:11434 30 | extra_hosts: 31 | - host.docker.internal:host-gateway 32 | restart: unless-stopped 33 | volumes: 34 | ollama-webui: {} 35 | ollama-intel-gpu: {} 36 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:24.04 2 | ENV DEBIAN_FRONTEND=noninteractive 3 | ENV TZ=america/los_angeles 4 | 5 | 6 | # Base packages 7 | RUN apt update && \ 8 | apt install --no-install-recommends -q -y \ 9 | software-properties-common \ 10 | ca-certificates \ 11 | wget \ 12 | ocl-icd-libopencl1 13 | 14 | # Intel GPU compute user-space drivers 15 | RUN mkdir -p /tmp/gpu && \ 16 | cd /tmp/gpu && \ 17 | wget https://github.com/oneapi-src/level-zero/releases/download/v1.21.9/level-zero_1.21.9+u24.04_amd64.deb && \ 18 | wget https://github.com/intel/intel-graphics-compiler/releases/download/v2.8.3/intel-igc-core-2_2.8.3+18762_amd64.deb && \ 19 | wget https://github.com/intel/intel-graphics-compiler/releases/download/v2.8.3/intel-igc-opencl-2_2.8.3+18762_amd64.deb && \ 20 | wget https://github.com/intel/compute-runtime/releases/download/25.09.32961.7/intel-level-zero-gpu_1.6.32961.7_amd64.deb && \ 21 | wget https://github.com/intel/compute-runtime/releases/download/25.09.32961.7/intel-opencl-icd_25.09.32961.7_amd64.deb && \ 22 | wget https://github.com/intel/compute-runtime/releases/download/25.09.32961.7/libigdgmm12_22.6.0_amd64.deb && \ 23 | dpkg -i *.deb && \ 24 | rm *.deb 25 | 26 | # Install Ollama Portable Zip 27 | ARG IPEXLLM_RELEASE_REPO=ipex-llm/ipex-llm 28 | ARG IPEXLLM_RELEASE_VERSON=v2.2.0 29 | ARG IPEXLLM_PORTABLE_ZIP_FILENAME=ollama-ipex-llm-2.2.0-ubuntu.tgz 30 | RUN cd / && \ 31 | wget https://github.com/${IPEXLLM_RELEASE_REPO}/releases/download/${IPEXLLM_RELEASE_VERSON}/${IPEXLLM_PORTABLE_ZIP_FILENAME} && \ 32 | tar xvf ${IPEXLLM_PORTABLE_ZIP_FILENAME} --strip-components=1 -C / && \ 33 | rm ${IPEXLLM_PORTABLE_ZIP_FILENAME} 34 | 35 | ENV OLLAMA_HOST=0.0.0.0:11434 36 | 37 | ENTRYPOINT ["/bin/bash", "/start-ollama.sh"] 38 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # *Update* 2 | 3 | Due to changing priorities, I no longer have time to dedicate maintaining this project. Hopefully the ipex team will support this natively in the future to make it easier for customers to use LLM technology on Intel products. See other repos like https://github.com/eleiton/ollama-intel-arc for alternate support. 4 | 5 | ----- 6 | 7 | # ollama-intel-gpu 8 | 9 | This repo illustrates the use of Ollama with support for Intel ARC GPU based via ipex-llm and Ollama Portable ZIP support. Run the recently released [deepseek-r1](https://github.com/deepseek-ai/DeepSeek-R1) model on your local Intel ARC GPU based PC using Linux 10 | 11 | ## Important Note 12 | 13 | All Ollama based ipex-llm defects should be reported directly to the ipex-llm project at https://github.com/intel/ipex-llm 14 | 15 | ## Screenshot 16 | ![screenshot](doc/screenshot.png) 17 | 18 | # Prerequisites 19 | * Ubuntu 24.04 or newer (for Intel ARC GPU kernel driver support. Tested with Ubuntu 24.04.02 20 | * Installed Docker and Docker-compose tools 21 | * Intel ARC series GPU (tested with Intel ARC A770 16GB and Intel(R) Core(TM) Ultra 5 125H integrated GPU) 22 | 23 | # Usage 24 | 25 | The following will build the Ollama with Intel ARC GPU support, and compose those with the public docker image based on OpenWEB UI from https://github.com/open-webui/open-webui 26 | 27 | Linux: 28 | ```bash 29 | $ git clone https://github.com/mattcurf/ollama-intel-gpu 30 | $ cd ollama-intel-gpu 31 | $ docker compose up 32 | ``` 33 | *Note:* On some platforms or versions of docker, the command above may instead be 'docker-compose'. 34 | 35 | *Note 2:* If you have multiple GPU's installed (like integrated and discrete), set the ONEAPI_DEVICE_DELECTOR environment variable in the docker compose file to select the intended device to use. 36 | 37 | Then launch your web browser to http://localhost:3000 to launch the web ui. Create a local OpenWeb UI credential, then click the settings icon in the top right of the screen, then select 'Models', then click 'Show', then download a model like 'llama3.1:8b-instruct-q8_0' for Intel ARC A770 16GB VRAM 38 | 39 | ## Update to the latest IPEX-LLM Portable Zip Version 40 | 41 | To update to the latest portable zip version of IPEX-LLM's Ollama, update the compose file with the build arguments shown below, using the latest `ollama-*.tgz` release from https://github.com/ipex-llm/ipex-llm/releases/tag/v2.3.0-nightly , then rebuild the image. 42 | 43 | ```yaml 44 | ollama-intel-gpu: 45 | build: 46 | context: . 47 | dockerfile: Dockerfile 48 | args: 49 | IPEXLLM_RELEASE_REPO: ipex-llm/ipex-llm 50 | IPEXLLM_RELEASE_VERSON: v2.3.0-nightly 51 | IPEXLLM_PORTABLE_ZIP_FILENAME: ollama-ipex-llm-2.3.0b20250415-ubuntu.tgz 52 | ``` 53 | 54 | # References 55 | * https://dgpu-docs.intel.com/driver/client/overview.html 56 | * https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quickstart/llamacpp_portable_zip_gpu_quickstart.md 57 | * https://github.com/intel/ipex-llm/releases/download/v2.2.0-nightly/ollama-ipex-llm-2.2.0b20250313-ubuntu.tgz 58 | --------------------------------------------------------------------------------