月色真美

月色真美

Windows平台安装Ollama并启用GPU

315
2024-07-27

需要科学上网，没条件的建议不要尝试了。

1.安装docker-desktop，使用wsl作为基础引擎。且安装Ubuntu-22.04 LTS，作为默认的wsl镜像

PS C:\Users\admin> wsl -l -v
  NAME                   STATE           VERSION
* Ubuntu-22.04           Running         2
  docker-desktop-data    Running         2
  docker-desktop         Running         2

2.安装GPU驱动（演示为nvidia显卡，AMD显卡自行查找资料安装）

2.1.配置apt源

root@admin:/mnt/c/Users/admin$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
[sudo] password for qilin:
deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/stable/deb/$(ARCH) /
#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/experimental/deb/$(ARCH) /

2.2.更新源

root@admin:/mnt/c/Users/admin$ sudo apt-get update
Get:1 https://nvidia.github.io/libnvidia-container/stable/deb/amd64  InRelease [1477 B]
Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease
Get:3 https://nvidia.github.io/libnvidia-container/stable/deb/amd64  Packages [10.1 kB]
Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB]
Get:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB]
Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1673 kB]
Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1888 kB]
Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [279 kB]
Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.0 kB]
Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2171 kB]
Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [337 kB]
Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.7 kB]
Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2238 kB]
Ign:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages
Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [384 kB]
Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [372 kB]
Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [572 B]
Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [604 B]
Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1108 kB]
Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [886 kB]
Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [258 kB]
Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [25.4 kB]
Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB]
Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB]
Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B]
Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB]
Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB]
Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B]
Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB]
Get:30 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB]
Get:31 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B]
Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2238 kB]
Get:32 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [173 kB]
Get:33 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [18.9 kB]
Get:34 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB]
Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7588 B]
Get:36 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B]
Fetched 12.4 MB in 59s (211 kB/s)
Reading package lists... Done

2.3.安装工具包

root@admin:/mnt/c/Users/admin$ sudo apt-get install -y nvidia-container-toolkit
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libnvidia-container-tools libnvidia-container1 nvidia-container-toolkit-base
The following NEW packages will be installed:
  libnvidia-container-tools libnvidia-container1 nvidia-container-toolkit nvidia-container-toolkit-base
0 upgraded, 4 newly installed, 0 to remove and 147 not upgraded.
Need to get 5583 kB of archives.
After this operation, 26.6 MB of additional disk space will be used.
Get:1 https://nvidia.github.io/libnvidia-container/stable/deb/amd64  libnvidia-container1 1.16.0-1 [924 kB]
Get:2 https://nvidia.github.io/libnvidia-container/stable/deb/amd64  libnvidia-container-tools 1.16.0-1 [19.8 kB]
Get:3 https://nvidia.github.io/libnvidia-container/stable/deb/amd64  nvidia-container-toolkit-base 1.16.0-1 [3512 kB]
Get:4 https://nvidia.github.io/libnvidia-container/stable/deb/amd64  nvidia-container-toolkit 1.16.0-1 [1128 kB]
Fetched 5583 kB in 4s (1547 kB/s)
Selecting previously unselected package libnvidia-container1:amd64.
(Reading database ... 27959 files and directories currently installed.)
Preparing to unpack .../libnvidia-container1_1.16.0-1_amd64.deb ...
Unpacking libnvidia-container1:amd64 (1.16.0-1) ...
Selecting previously unselected package libnvidia-container-tools.
Preparing to unpack .../libnvidia-container-tools_1.16.0-1_amd64.deb ...
Unpacking libnvidia-container-tools (1.16.0-1) ...
Selecting previously unselected package nvidia-container-toolkit-base.
Preparing to unpack .../nvidia-container-toolkit-base_1.16.0-1_amd64.deb ...
Unpacking nvidia-container-toolkit-base (1.16.0-1) ...
Selecting previously unselected package nvidia-container-toolkit.
Preparing to unpack .../nvidia-container-toolkit_1.16.0-1_amd64.deb ...
Unpacking nvidia-container-toolkit (1.16.0-1) ...
Setting up nvidia-container-toolkit-base (1.16.0-1) ...
Setting up libnvidia-container1:amd64 (1.16.0-1) ...
Setting up libnvidia-container-tools (1.16.0-1) ...
Setting up nvidia-container-toolkit (1.16.0-1) ...
Processing triggers for libc-bin (2.35-0ubuntu3.1) ...
/sbin/ldconfig.real: /usr/lib/wsl/lib/libcuda.so.1 is not a symbolic link

3.使用docker-compose安装Ollama

  ollama:
    image: ollama/ollama:0.3.0
    container_name: ollama
    tty: true
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    ports:
      - "11434:11434"
    volumes:
      - D:\mydata\docker\ollama:/root/.ollama
    restart: always
    networks:
      - backend
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    depends_on:
      - ollama
    ports:
      - "3000:8080"
    volumes:
      - D:\mydata\docker\open-webui\data:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - WEBUI_SECRET_KEY=
    restart: always
    networks:
      - backend

脚本中volumes替换成自己的路径

脚本中使用的是Nvidia显卡，如果是AMD显卡，请参考官网说明进行变更https://docs.docker.com/compose/gpu-support

4.启动编排脚本

5.进入open-webui

6.注册一个账号，本地账号，随意注册即可

7.进入管理员面板

8.下载一个大模型，这里我下载qwen2测试

更多模型进入ollama的大模型库，复制大模型的名字输入到拉取位置即可，大模型库：https://ollama.com/library

也可以自行进入ollama的bash里，使用ollama的run命令进行拉取

9.选择下载的大模型，开始使用

PS：我的战五渣RTX3050，太过惨烈

某种物质
搭建全平台私有远程控制RustDesk