How to install GPT Gateway

GPT Gateway Fresh Install Guide

1. Setting Up a Hostname:

   • Create a hostname for the GPT gateway API.

   • Example: https://bgdXgateway.agatdemo.com.

2. Prepare an Ubuntu Machine:

   • Provision an Ubuntu machine for the GPT gateway.

   • Open the following ports:

     • 22 (SSH)

     • 443 (HTTPS)

     • 1433 (SQL Server)

     • 5432 (PostgreSQL)

3. Disable ipv6 (there are some ways, look online)

4. Obtaining the GPT Gateway Version:

   • Login to https://gitlab.com/agat-software/python/gatewaychatgpt/-/releases with user itAgat (Lastpass)

   • Download the last zip

     Open

     

   • Extract and browse gatewaychatgpt-xxx\gatewaychatgpt-xxx\project\config

5. Configuration Changes:

   a. docker.env:

Modify environment variables as needed.

• For private AI set LOCAL_AI_API=http://<LLM-ip:port>

• For OpenAI set MODEL = gpt-3.5-turbo-16k , For private AI set MODEL=meta-llama/Meta-Llama-3.1-8B-Instruct

• If you are using OpenAI - Create API key https://agatsoftware.atlassian.net/wiki/spaces/SKYP/pages/2991423496 and use it in line 25 - OPENAI_API_KEY= sk-W…..

• For on-prem environment you can use the general client id and secret in Lastpass “Bgd Gateway general client”

• For SaaS environment generate a client ID and secret here: Free Online GUID Generator and document in your BGD Lastpass record

• EMBEDDER = (Set openai for GPT or gte-qwen for Local AI)

• Database

b. gateway.conf:

1. This is a NGINX configuration file.

2. Update the server_name property (ports 80 and 443) with your gateway URL.

3. Example:

   server_name bgdXgateway.agatdemo.com www.bgdXgateway.agatdemo.com;

c. postgresql.conf

• This is a PostgreSQL configuration file.

• Use PGTune to configure PostgreSQL depending on your machine. Read this: Deployment system sizing

When using Private AI, move the file compose.override.yml from folder Private AI to folder Gateway.

d. docker-compose-app.service

• edit WorkingDirectory=/home/user/Gateway - update with the proper user

e. Edit SQLScripts/FreshInstall/CreateDatabase.sql

• Set dbName, dbUserName and dbUserPassword

Move to Gateway server

1. Upload to the Ubuntu Machine: (Be aware of case sensitivity in Linux)

   1. Transfer the gateway folder to the Ubuntu machine's home directory (i.e., ~).

   2. Tip: Use the MobaXterm tool's UI for an SSH connection to assist with the transfer.

2. Installation and Setup on the Ubuntu Machine:

   • Connect to the machine and navigate to the home directory:

     cd ~

   • Resolving Windows to Unix Copy Issues:

     sudo apt install -y dos2unix sudo dos2unix gateway/InstallScripts/FreshInstall/install.sh sudo dos2unix gateway/gateway.conf

   • Install docker - Documentation here (use apt method) [Note that Nvidia deep learning base AMI already included docker]

   • Install docker compose plugin - Documentation here (use apt method)

   • Login to Docker BusinessGPT account (Password in LP):

   • Grant execution permissions and run the installation script (login to docker account using “Docker BusinessGPT” in LP): (case sensitive)

3. Installing Gateway database:

Connect to the database with SSMS from office (with ip address of the Gateway)

username: sa, password - see sa password in docker.env

Run CreateDatabase.sql (take from Sqlsripts\freshinstall)

4. Install service docker-compose-app.service

edit user and run:

then run:

5. Health Check:

   • Browse to: {DOMAIN}/v1/alive (replace {DOMAIN} with your hostname).

   • Ensure the result displays: “Yes”.

Open

Troubleshooting

If you change anything in docker.env or docker.compose, run this to restart containers

Ubuntu

Amazon Linux

Check gateway docker status:

Ubuntu

Amazon Linux

Check gateway logs:

Ubuntu:

Amazon Linux:

Check system logs:

Ubuntu:

Amazon Linux:

Check GPU memory (for Private AI)

If PostgreSQL not configured in docker.env and you already run docker compose - you will need to delete PostgreSQL volume (this will delete all embedding)

Ubuntu:

Amazon Linux:

How to find Gateway version

To locate the version of the Gateway within your Gateway container, navigate to the /gateway/main.py file. You can retrieve the Gateway version by executing the following command:

Import logs from docker to current location:

First locate location of relevant log

You might need logs from these locations:

gateway_gateway-data

gateway_doc-loader-data

Copy to current location example: