doru/intelligence-router
1
0
Fork 0
Code Issues 17 Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: doru:555a887b4e3844e410bdc7b65f513cc17c5c5209
Branches Tags
doru:master
doru:feature/add-model-profiles
..
compare: doru:b7079fa1993bd6ece9c59dd1b122f88f9e27a5cf
Branches Tags
doru:master
doru:feature/add-model-profiles

2 Commits
555a887b4e ... b7079fa199

Author SHA1 Message Date
Tudorel Oprisan
b7079fa199 fixed port and conflict 2026-06-15 14:07:18 +01:00
root
e14d2c62da fix: use venv for sidecar deps, add missing deploy steps 
- llm-sidecar.service: use /home/bigt/AI/llm/venv/bin/uvicorn instead of
  global python3 -m uvicorn (avoids 'No module named uvicorn' error)
- deploy/README.md: add steps to copy sidecar/ package, create venv,
  and pip install requirements.txt
 2026-06-15 13:02:34 +00:00
2 changed files with 17 additions and 7 deletions
Show Stats Expand all files Collapse all files

18
deploy/README.md
View File

@ -8,22 +8,32 @@ On the Main PC:
# 1. Copy the service file # 1. Copy the service file
sudo cp deploy/llm-sidecar.service /etc/systemd/system/ sudo cp deploy/llm-sidecar.service /etc/systemd/system/
# 2. Copy the manifest (adjust paths as needed) # 2. Create the working directory and copy files
mkdir -p /home/bigt/AI/llm mkdir -p /home/bigt/AI/llm
cp deploy/manifest.yaml /home/bigt/AI/llm/manifest.yaml cp deploy/manifest.yaml /home/bigt/AI/llm/manifest.yaml
# 3. Create a .env for the sidecar (optional) # Copy the sidecar Python package (app.py + manifest.py)
cp -r sidecar/ /home/bigt/AI/llm/sidecar/
# Copy requirements.txt for the venv
cp requirements.txt /home/bigt/AI/llm/
# 3. Create a Python virtual environment with dependencies
python3 -m venv /home/bigt/AI/llm/venv
/home/bigt/AI/llm/venv/bin/pip install -r /home/bigt/AI/llm/requirements.txt
# 4. Create a .env for the sidecar (optional)
cat > /home/bigt/AI/llm/.env << 'EOF' cat > /home/bigt/AI/llm/.env << 'EOF'
# Sidecar configuration # Sidecar configuration
MANIFEST_PATH=/home/bigt/AI/llm/manifest.yaml MANIFEST_PATH=/home/bigt/AI/llm/manifest.yaml
SIDECAR_PORT=8081 SIDECAR_PORT=8081
EOF EOF
# 4. Enable and start the service # 5. Enable and start the service
sudo systemctl daemon-reload sudo systemctl daemon-reload
sudo systemctl enable --now llm-sidecar sudo systemctl enable --now llm-sidecar
# 5. Verify it's running # 6. Verify it's running
sudo systemctl status llm-sidecar sudo systemctl status llm-sidecar
``` ```

6
deploy/llm-sidecar.service
View File

@ -11,10 +11,10 @@ WorkingDirectory=/home/bigt/AI/llm
EnvironmentFile=-/home/bigt/AI/llm/.env EnvironmentFile=-/home/bigt/AI/llm/.env
Environment=MANIFEST_PATH=/home/bigt/AI/llm/manifest.yaml Environment=MANIFEST_PATH=/home/bigt/AI/llm/manifest.yaml
Environment=SIDECAR_PORT=8081 Environment=SIDECAR_PORT=8081
Environment=PATH=/usr/local/bin:/usr/bin:/bin Environment=PATH=/home/bigt/AI/llm/venv/bin:/usr/local/bin:/usr/bin:/bin
# Executable — adjust path as needed # Use the sidecar's venv — install deps via deploy/README.md
ExecStart=/usr/bin/python3 -m uvicorn sidecar.app:app --host 0.0.0.0 --port 8080 ExecStart=/home/bigt/AI/llm/venv/bin/uvicorn sidecar.app:app --host 0.0.0.0 --port 8080
Restart=always Restart=always
RestartSec=3 RestartSec=3