Skip to content

Project Setup Guide

This document provides comprehensive step-by-step instructions to deploy Infosys Agentic Foundry in Azure Kubernetes Cluster(AKS).

Prerequisites

Before setting up the project, ensure you have the following requirements and access permissions:

Ensure users have access to below:

  • Python 3.12+ - Required Python version

  • React - Frontend framework

  • Infosys Github Repo: Infosys-Agentic-Foundry

  • Ensure users must have their Azure OpenAI Keys and Endpoints, Speech to text Key and Endpoint.
  • Ensure Azure resources are created, and you have access/connectivity to push and pull from these resources
    • Azure Container Registry (ACR)
    • Azure Kubernetes Service (AKS)
    • Azure Postgres Service (Compute size: Standard_B2s (2 vCores, 4 GiB memory, 1280 max iops), Storage: 32 GiB)
    • Azure Linux Virtual Machine
  • Install Docker, Azure CLI, kubectl in your Azure VM
  • Create a namespace in the AKS cluster as per your requirement (Optional)
kubectl create namespace <namespace>

Azure Deployment

ARIZE PHOENIX

STEPS FOR DEPLOYING ARIZE PHOENIX IN AKS

  1. Create a yaml file for deploying arize phoenix as a container, you can use the arize phoenix image in the yaml file.

    nano filename1.yaml

    Info

    You can get the image from Docker - Phoenix or any other trusted source which your organization allows

  2. While creating YAML file, please include these aspects as well in your script,

    • Include a Deployment definition with replica count and container configuration
    • Define multiple container ports for HTTP, gRPC, and metrics exposure
    • Configure environment variables for database connectivity and application settings
    • Add proper labels and selectors to ensure correct pod and service mapping
    • Include volume mounts and volumes for application storage requirements
    • Define a Service resource to expose the application externally or internally
    • Configure multiple service ports to match container ports
    • Use a LoadBalancer service type with internal access configuration or choose as per your environment.
    • Specify a namespace for isolation and organization
    • Specify resource requests and limits for CPU and memory (Recommended configuration, requests: ( cpu: "500m", memory: "1Gi") and limits: ( cpu: "1CPU", memory: "2Gi") )
    • Consider adding secure handling of sensitive data, health probes, and scaling configurations for production readiness

  3. Now you need to use this command for creating deployment and service:

    kubectl apply -f filename1.yaml

  4. You can check the pods deployed using the command below

    kubectl get pods -n namespace

  5. You can check the services deployed using the command below

    kubectl get svc -n namespace

  6. Note down the load balancer IP for the container. You need to update it in the .env of your backend and frontend folders before creating the respective docker images.

REDIS

STEPS FOR DEPLOYING REDIS IN AKS

  1. Create a yaml file for deploying Redis as a container, you can use the redis image in the yaml file.

    nano filename2.yaml

    Info

    You can get the image from Image Layer Details - redis:8.2.1 or any other trusted sources which your organization allows

  2. While creating YAML file, please include these aspects as well in your script,

    • Include a Deployment definition with replica count and container configuration
    • Configure container command arguments for enabling authentication (e.g., password protection)
    • Define the container port used by the application (Redis default port 6379)
    • Set environment variables for host, port, database index, password, and cache settings
    • Specify resource requests and limits to manage CPU and memory usage memory (Recommended configuration, requests: ( cpu: "300m", memory: "512Mi") and limits: ( cpu: "1CPU", memory: " 2Gi") )
    • Add proper labels and selectors for linking Deployment and Service
    • Define a Service resource to expose the Redis application
    • Configure the Service port mapping to match the container port
    • Use a LoadBalancer service type with internal access configuration or choose as per your environment
    • Specify a namespace for isolation and organization
    • Consider adding secure credential management, persistence (volumes), and scaling strategies for production environments

  3. Now you need to use this command for creating deployment and service: 

    kubectl apply -f filename2.yaml

  4. You can check the pods deployed using the command below 
    kubectl get pods -n namespace
  5. You can check the services deployed using the command below 
    kubectl get svc -n namespace
  6. Note down the load balancer IP for the container. You need to update it in the .env of your backend folder before creating the respective docker image.

GRAFANA

STEPS FOR DEPLOYING GRAFANA IN AKS

  1. Create a yaml file for deploying Grafana as a container, you can use the grafana image in the yaml file. 

    nano filename3.yaml

    Info

    You can get the image from Image Layer Details - grafana/grafana:11.2.0 or any other trusted sources which your organization allows

  2. While creating YAML file, please include these aspects as well in your script,

    • Include a Deployment definition with container image, replica count, and pod configuration
    • Specify resource requests and limits for CPU and memory(Recommended configuration, requests: ( cpu: "200m", memory: "256Mi") and limits: ( cpu: "500m", memory: "512Mi") )
    • Configure environment variables for application settings
    • Define the container port used by the application
    • Add proper labels and selectors to link Deployment and Service
    • Include a Service definition with type, ports, and target pod mapping
    • Use a namespace for isolation and organization
    • Use a LoadBalancer service type with internal access configuration or choose as per your environment
    • Optionally include Secrets, readiness/liveness probes, and scaling settings for production readiness

  3. Now you need to use this command for creating deployment and service: 

    kubectl apply -f filename3.yaml

  4. You can check the pods deployed using the command below 
    kubectl get pods -n namespace
  5. You can check the services deployed using the command below 
    kubectl get svc -n namespace
  6. Note down the load balancer IP for the container. You need to update it in the .env of your backend and frontend folders before creating the respective docker images.

STEPS FOR DEPLOYING ELASTIC SEARCH IN AKS

  1. Create a yaml file for deploying elastic search as a container, you can use the elastic search image in the yaml file. 

    nano filename4.yaml

    Info

    You can get the image from elasticsearch - Official Image | Docker Hub or any other trusted sources which your organization allows

  2. While creating YAML file, please include these aspects as well in your script,

    • Include a Namespace definition for logical isolation of resources
    • Use a StatefulSet instead of a Deployment for managing stateful applications like Elasticsearch
    • Define replica count and stable network identity for pod management
    • Configure multiple container ports for HTTP (9200) and internal transport (9300)
    • Set environment variables for cluster configuration, memory settings, and security options
    • Specify resource requests and limits for CPU and memory to handle Elasticsearch workload (Recommended configuration, requests: ( cpu: "1CPU", memory: "3Gi") and limits: ( cpu: "2CPU", memory: " 6Gi") )
    • Configure volume mounts and volumes for data storage
    • Use persistent storage considerations (even if temporary storage is used in simple setups)
    • Add proper labels and selectors to connect StatefulSet and Service
    • Define a Service resource to expose the application
    • Configure multiple service ports to match container ports
    • Use a LoadBalancer service type with internal access configuration or choose as per your environment
    • Ensure namespace consistency across all resources
    • Consider adding persistent volumes, security settings, and scaling strategies for production readiness

  3. Now you need to use this command for creating deployment and service: 

    kubectl apply -f filename4.yaml

  4. You can check the pods deployed using the command below 
    kubectl get pods -n namespace
  5. You can check the services deployed using the command below 
    kubectl get svc -n namespace
  6. Note down the load balancer IP for the container and update it in the OpenTelemetry YAML script.

OPEN-TELEMETRY

STEPS FOR DEPLOYING OPEN-TELEMETRY IN AKS

  1. Create a yaml file for deploying OpenTelemetry as a container, you can use the OpenTelemetry directly in the yaml file. 

    nano filename5.yaml

    Info

    You can get the image from otel/OpenTelemetry-collector-contrib - Docker Image or any other trusted sources your organization allows

  2. While creating YAML file, please include these aspects as well in your script,

    • Include a Namespace definition to logically isolate monitoring components
    • Use a ConfigMap to store OpenTelemetry Collector configuration (receivers, exporters, processors, pipelines)
    • Define receivers for OTLP protocols (gRPC and HTTP) to ingest telemetry data
    • Configure exporters for debugging and external systems (e.g., Elasticsearch)
    • Add processors (such as batch) to optimize telemetry handling
    • Define service pipelines for traces, metrics, and logs with appropriate receivers, processors, and exporters
    • Include a Deployment definition for running the OpenTelemetry Collector
    • Configure container arguments to load the external configuration file from the ConfigMap
    • Define container ports for OTLP gRPC and HTTP ingestion
    • Use volume mounts and volumes to inject ConfigMap data into the container
    • Add proper labels and selectors to connect Deployment and Service
    • Define a Service resource to expose the collector endpoints
    • Configure multiple service ports matching OTLP protocols
    • Use a LoadBalancer service type with internal access configuration or choose as per your environment
    • Ensure namespace consistency across all resources
    • Consider adding secure endpoints, resources, scaling, and advanced processors/exporters for production readiness (Recommended configuration, requests: ( cpu: "500m", memory: "1CPU") and limits: ( cpu: "512Mi", memory: " 1Gi") )

  3. Now you need to use this command for creating deployment and service: 

    kubectl apply -f filename5.yaml

  4. You can check the pods deployed using the command below 
    kubectl get pods -n namespace

  5. You can check the services deployed using the command below 

    kubectl get svc -n namespace

  6. Note down the load balancer IP for the container. You need to update it in the .env of your backend folder before creating the respective docker image.

MODEL SERVER

STEPS TO SETUP MODEL SERVER

For detailed instructions on deploying and configuring your model server, refer to the Model Server Deployment guide.

Note

You need to update the URL for the model server in the .env of the backend folder

BACKEND

STEPS TO SETUP BACKEND

Download Backend code from GitHub. For detailed instructions, see Download the Backend Project Code.

  1. Login in to Azure VM
  2. Download Backend source code folder into Azure VM
  3. Before starting backend image creation make sure that arize phoenix, OpenTelemetry, Grafana, Elastic Search, Redis and Model server are setup as per instructions provided above. Update the respective urls of all these services in .env file.
  4. Update the remaining values for the variables in the .env file.
  5. Change working directory to the Backend Folder: 
    cd `<BE foldername>`
  6. Create Dockerfile inside Backend folder to create docker image for Backend code.
  7. Create backend image: 
        docker build -f `<dockerfile-name>` -t `<tag-name>`
  8. Retag the created image to ACR name: 
    docker tag localhost/<imagename>:<tag> <acr login server>/<imagename>:<tag>
  9. Login to az and then login to acr: 
    docker login <acr login servername>
  10. Push the retagged image to ACR: 
    docker push <acr login servername>/<imagename>:<tag>
  11. Create backend deployment file: 
    nano <deployment filename.yaml>

  12. While creating YAML file, please include these aspects as well in your script,

    • Include a Deployment definition with replica count and container configuration
    • Specify the container image pulled from a private container registry
    • Define the container port used by the application
    • Add proper labels and selectors to ensure correct mapping between pods and services
    • Optionally configure environment variables for application-specific settings
    • Define a Service resource to expose the application within or outside the cluster
    • Configure service port mapping to forward traffic to the container port
    • Use a LoadBalancer service type with internal access configuration or choose as per your environment
    • Specify a namespace for isolation and resource organization
    • Consider adding resource limits, health probes, and scaling configurations for production readiness (Recommended configuration, requests: ( cpu: "250m", memory: "1Gi") and limits: ( cpu: "500m", memory: " 1.5Gi") )

  13. Login to Azure Kubernetes

  14. Execute the deployment file: 
    kubectl apply -f <deployment filename.yaml>
  15. Check if the pods is deployed successfully: 
    kubectl get pods -n <namespace>
  16. Check if the service is up & running successfully: 
    kubectl get svc -n <namespace>

FRONTEND

STEPS TO SETUP FRONTEND

Download Frontend code from GitHub. For detailed instructions, see Download the Frontend Project Code.

  1. Login in to Azure VM
  2. Download Frontend source code folder into Azure VM
  3. Before starting frontend image creation make sure that arize phoenix, Grafana are setup as per instructions provided above. Update the respective urls of all these services in .env file.
  4. Change working directory to the Frontend Folder: 
        cd <Frontend foldername>
  5. Create Dockerfile inside Frontend folder to create docker image from Frontend code.

  6. In .env file of Frontend, update below service loadbalancer urls which were generated after deployment of backend server in Azure Kubernetes.

    • REACT_APP_BASE_URL (use Backend url),
    • REACT_APP_MKDOCS_BASE_URL (use mkdocs url),
    • REACT_APP_LIVE_TRACKING_URL (use Arize phoenix url),
    • REACT_APP_GRAFANA_DASHBOARD_URL (use Grafana url)

  7. Create Frontend image: 

        docker build -f <dockerfile-name> -t <tag-name>

  8. Retag the created image to ACR name: 
    docker tag localhost/<imagename>:<tag> <acr login server>/<imagename>:<tag>
  9. Login to az and then login to acr: 
    docker login <acr login servername>
  10. Push the retagged image to ACR: 
    docker push <acr login servername>/<imagename>:<tag>
  11. Create Frontend deployment file: 
    nano <deployment filename.yaml>

  12. While creating YAML file, please include these aspects as well in your script,

    • Include a Deployment definition with replica count and container configuration
    • Specify the container image pulled from a private container registry
    • Define the container port used by the application
    • Add proper labels and selectors to ensure correct mapping between pods and services
    • Optionally configure environment variables for application-specific settings
    • Define a Service resource to expose the application within or outside the cluster
    • Configure service port mapping to forward traffic to the container port
    • Use a LoadBalancer service type with internal access configuration or choose as per your environment
    • Specify a namespace for isolation and resource organization
    • Consider adding resource limits, health probes, and scaling configurations for production readiness (Recommended configuration, requests: ( cpu: "250m", memory: "512Mi") and limits: ( cpu: "1CPU", memory: " 1Gi") )

  13. Login to Azure Kubernetes

  14. Execute the deployment file: 
    kubectl apply -f <deployment filename.yaml>
  15. Check if the pods is deployed successfully: 
    kubectl get pods -n <namespace>
  16. Check if the service is up & running successfully: 
    kubectl get svc -n <namespace>

Troubleshooting

Virtual Environment Activation Fails

  • Permissions Error: Try running command prompt as administrator

Dependency Installation Errors

Update pip to the latest version: 

python -m pip install --upgrade pip

Server or UI Not Starting

  1. Verify the virtual environment is active
  2. Check for typos in commands or file names
  3. Ensure all dependencies are properly installed
  4. To troubleshoot check pod logs in Kubernetes, By using below commands: 
    kubectl describe pods <pod_name> -n <namespace>
kubectl logs <pod_name> -n <namespace>