Skip to content

Deployment of Debugging Interface and API

Jump to bottom
supinyu edited this page Jun 20, 2024 · 8 revisions
  • [Manual Deployment](# 1、Manual Deployment)
  • CDK

1、Manual Deployment

1. Prepare EC2 Instance

Create an EC2 with following configuration:

- OS Image (AMI): Amazon Linux 2023, Amazon Linux 2(AL2 End of Life is 2025-06-30)
- Instance type: t3.large or higher
- VPC: use default one and choose a public subnet
- Security group: Allow access to 22, 80, 8000 port from anywhere (Select "Allow SSH traffic from Anywhere" and "Allow HTTP traffic from the internet")
- Storage (volumes): 1 GP3 volume(s) - 30 GiB

2. Config Permission

2.1 IAM Role's permission

Create a new IAM role with name genbirag-service-role and settings below:

  • Trusted entity type: AWS Service
  • Service: EC2
  • Use Case: EC2 - Allows EC2 instances to call AWS services on your behalf.

Skip "Add permission" and create this role first.

After the role is created, and then add permission by creating inline policy as below:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel",
                "bedrock:InvokeModelWithResponseStream",
                "dynamodb:*Table",
                "dynamodb:*Item",
                "dynamodb:Scan",
                "dynamodb:Query"
            ],
            "Resource": [
                "arn:aws:bedrock:us-west-2::foundation-model/*",
                "arn:aws:dynamodb:us-west-2:**YOURACCOUNTID**:table/Nlq*"
            ]
        }
    ]
}

Finally, Bind this IAM instance profile (IAM Role) to your EC2 instance.

You can refer to the EC2 document - Using IAM roles

2.2 Amazon Bedrock's Model Permission

Make sure you have enabled model access in AWS Console in us-west-2 (Oregon) region for Anthropic Claude model and Amazon Titan embedding model.

image

3. Install Docker and Docker Compose

Log in to the EC2 instance using SSH command as the ec2-user user or use the AWS EC2 Instance Connect feature in the EC2 console to log in to the command line.

In the session, execute the following commands. Note: Execute each command one line at a time.

If you are not this user, you can switch with the following command:

sudo su - ec2-user
# Install components
sudo dnf install docker python3-pip git -y && pip3 install -U awscli && pip3 install docker-compose

# For Amazon Linux 2，use yum to replace dnf

sudo yum install docker python3-pip git -y && pip3 install -U awscli && sudo pip3 install docker-compose

# Fix docker python wrapper 7.0 SSL version issue  
pip3 install docker==6.1.3 

# Fix requests version issue 
pip3 install requests==2.31.0

# Configure components
sudo systemctl enable docker && sudo systemctl start docker && sudo usermod -aG docker $USER

# Exit the terminal
exit

4. Install the Demo Application

Reopen a terminal session and continue executing the following commands:

Note: Execute each command one line at a time.

# Log in as user ec2-user

# Configure OpenSearch server parameters
sudo sh -c "echo 'vm.max_map_count=262144' > /etc/sysctl.conf" && sudo sysctl -p

# Clone the code
git clone https://github.com/aws-samples/generative-bi-using-rag.git

# Config the Environment Variable in .env file, modify AWS_DEFAULT_REGION to the region same as the EC2 instance.
cd generative-bi-using-rag/application && cp .env.template .env

Configure login username and password. Before deploying the code, the login username and password must be set

Modify the following file:

application/config_files/stauth_config.yaml

This is an example.

credentials:
  usernames:
    jsmith:
      email: [email protected]
      name: John Smith
      password: abc # To be replaced with hashed password
    rbriggs:
      email: [email protected]
      name: Rebecca Briggs
      password: def # To be replaced with hashed password
cookie:
  expiry_days: 30
  key: random_signature_key # Must be string
  name: random_cookie_name
preauthorized:
  emails:
  - [email protected]

The password needs to be converted from plaintext to a hashed password, which can be obtained through the following methods

pip install streamlit-authenticator~=0.3.2

python3 generate_streamlit_password.py

Enter the password that needs to be converted to obtain the hashed password, and add the password change to the application/config_files/stauth_config.yaml file

please enter the password: abc

hashed_passwords:  xxxxxxxxxxxx

Compile files and build Docker images


# Build docker images locally
docker-compose build

# Start all services
docker-compose up -d

# Wait 3 minutes for MySQL and OpenSearch to initialize
sleep 180

If the code has been modified and needs to be recompiled, it can be executed

bash docker-compose-build.sh

5. Initialize MySQL

In the terminal, continue executing the following commands::

cd initial_data && wget https://github.com/fengxu1211/generative-bi-using-rag/raw/demo_data/application/initial_data/init_mysql_db.sql.zip

unzip init_mysql_db.sql.zip && cd ..

docker exec nlq-mysql sh -c "mysql -u root -ppassword -D llm  < /opt/data/init_mysql_db.sql"

6. Initialize Amazon OpenSearch docker version

6.1 Initialize the index for the sample data by creating a new index

docker exec nlq-webserver python opensearch_deploy.py

If the command fails due to any errors, delete the index and rerun the previous command:

curl -XDELETE -k -u admin:admin "https://localhost:9200/uba"

6.2 (Optional) Bulk import custom QA data by appending to an existing index:

docker exec nlq-webserver python opensearch_deploy.py custom false

7. Access the Streamlit Web UI

Open in your browser: http://<your-ec2-public-ip>

Note: Use HTTP instead of HTTPS.

Enter the set username and password

username: admin
password:

8. Access the API

Open in your browser: http://<your-ec2-public-ip>:8000

Note: Use HTTP instead of HTTPS.

2、CDK （coming soon）

Clone this wiki locally