Rockylinux9.x 下本地部署大模型及知识库(ollama+docker+ragflow)

1.安装docker版ragflow

安装Docker CE

dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo

dnf update

dnf install -y docker-ce

替换docker源

1

vim /etc/docker/daemon.json

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


{
"registry-mirrors":
[
"https://docker.1ms.run",
"https://docker.xuanyuan.me",
"https://fast360.xyz",
"https://docker.m.daocloud.io",
"https://1ms.run",
"https://docker.zhai.cm",
"https://docker.gbfeng.com",
"https://hub.haod.eu.org",
"https://xdark.top",
"https://docker.1ms.run",
"https://registry.docker-cn.com",
"https://docker.registry.cyou",
"https://docker-cf.registry.cyou",
"https://dockerpull.com",
"https://dockerproxy.cn",
"https://docker.1panel.live",
"https://hub.rat.dev",
"https://docker.anyhub.us.kg",
"https://docker.chenby.cn",
"https://dockerhub.icu",
"https://docker.awsl9527.cn",
"https://dhub.kubesre.xyz",
"https://docker.hlyun.org",
"https://docker.m.daocloud.io"
]
}

重启docker

1
2


systemctl enable docker
systemctl restart docker

将ragflow的docker文件夹上传至/home/ragflow，编辑docker文件夹中的.env,注释掉默认的轻量版,取消完整版注释，取消HF镜像注释

1
2
3


# RAGFLOW_IMAGE=infiniflow/ragflow:v0.16.0-slim
RAGFLOW_IMAGE=infiniflow/ragflow:v0.16.0
HF_ENDPOINT=https://hf-mirror.com

在/home/ragflow/docker目录下执行启动命令，拉取docker镜像

docker compose -f docker-compose.yml up -d

2.安装ollama

手动安装ollama

删除旧库

sudo rm -rf /usr/lib/ollama

解压ollama

sudo tar -C /usr -xzf ollama-linux-amd64.tgz

启动 Ollama

ollama serve

在另一个终端中，验证 Ollama 是否正在运行：

1

ollama -v

为 Ollama 创建用户和组

1
2


sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama
sudo usermod -a -G ollama $(whoami)

创建服务文件：/etc/systemd/system/ollama.service

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


[Unit]
Description=Ollama Service
After=network-online.target

[Service]
ExecStart=/usr/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=$PATH"
Environment="OLLAMA_HOST=0.0.0.0"
#允许所有ip访问ollama,否则默认只允许本地访问
Environment="OLLAMA_MODELS=/home/ollama/models"
#如果有上一行参数指定模型存放路径，请参照授权chown -R ollama:ollama /home/ollama/models

[Install]
WantedBy=default.target

启动服务

1
2


sudo systemctl daemon-reload
sudo systemctl enable ollama

启动ollama并验证

1
2


sudo systemctl start ollama
sudo systemctl status ollama

查看日志

1

journalctl -e -u ollama

如需要卸载

删除 ollama 服务：

1
2
3


sudo systemctl stop ollama
sudo systemctl disable ollama
sudo rm /etc/systemd/system/ollama.service

从 bin 目录中删除 ollama 二进制文件（、、或）：/usr/local/bin``/usr/bin``/bin

1

sudo rm $(which ollama)

删除下载的模型以及 Ollama 服务用户和组：

1
2
3


sudo rm -r /usr/share/ollama
sudo userdel ollama
sudo groupdel ollama

删除已安装的库：

1

sudo rm -rf /usr/local/lib/ollama