GPT-4 to LLaMA2 on AWS migration for Enterprise SaaS Analytics -

GPT-4 to LLaMA2 on AWS migration for Enterprise SaaS Analytics

Business Problem

  • Lyzr is an Antler backed startup, which builds building blocks & Generative AI powered products. NeoAnalyst, one solution from Lyzr, is an AI data analyst platform, which enables business leaders to interact with data through natural language queries. The existing solution was built on GPT-4. The target audience for the platform being enterprise business users, they faced multiple challenges in enterprise deployments, as they scaled:

    Enterprise customer preferences for an open LLM model over OpenAI’s GPT series
    Ensure the entire platform remains compliant with.
    GDPR regulations, ensuring user data privacy and protection.
    SOC2 standards, guaranteeing security, availability, processing integrity, confidentiality, and privacy of customer data.

    Additionally, the NeoAnalyst product required the implementation of an agent-based model for facilitating interactions between multiple agents that collectively provide AI data analyst functionality. 


Lyzr is an enterprise Generative AI company that offers private and secure AI Agent SDKs and an AI Management System. Lyzr helps enterprises build, launch and manage secure GenAI applications, in their AWS cloud or on-prem infra. No more sharing sensitive data with SaaS platforms or GenAI wrappers. And no more reliability and integration issues of open-source tools.


Post evaluation by the GoML Data Scientists, LLaMA2 was the target LLM for migration, which met the above-mentioned criteria. The migration process involved transitioning the existing chatbot system from using ChatGPT’s capabilities by leveraging the enhanced features of LLaMA2.  

The overall scope identified by the goML’s GenAI team was broken into 3 parts:

GPT-4 to LLaMA2 on AWS migration for Enterprise SaaS Analytics -
Click to View in Full Size


  • User Interface and Routing: Frontend application for user interaction, managed by Amazon Route 53 for reliable DNS routing.
  • API Gateway and Load Balancer: Amazon API Gateway processes API requests, and Elastic Load Balancer distributes traffic across multiple EC2 instances for high availability.
  • Compute and Serverless Processing: Amazon EC2 instances and AWS Fargate host backend applications, while AWS Lambda handles serverless function execution.
  • Data Storage and Management: Amazon S3 for object storage, Amazon RDS for relational databases, Amazon DynamoDB for NoSQL storage, and Amazon Redshift for data warehousing.
  • Search and Analytics: Amazon Elasticsearch Service provides real-time search and analytics capabilities.
  • Machine Learning and Data Integration: Amazon SageMaker for machine learning model development and deployment, and AWS Glue for data integration and preparation.
  • Security and Monitoring: AWS IAM for secure access management, AWS CloudWatch for monitoring and observability, and AWS WAF for web application security.


Increased conversion for inbound queries, with increased relevance of property suggestions


Reduced response time
to inbound queries


Overall increased efficiency of the sales engine, with the auto co-pilot

Technology Stack​