Mariana.AI is an innovative healthcare company based on automating clinical documentation and patient engagement. GoML partnered with Mariana.AI to upgrade its clinical note generation and medical coding workflows from OpenAI to AWS Bedrock. To improve performance, prompt engineering and establish a scalable, compliant backend for future expansion into real-time and voice-based clinical support was the goal.
The problem: OpenAI based AI clinical notes that impacted accuracy
Mariana.AI had an OpenAI based deployment which generated AI clinical notes and medical codes from transcribed doctor-patient conversations. When it comes to accuracy, tone, and clinical relevance, the system faces limitations. Also, the team wanted to explore different vendors and choose an option that provided better flexibility and long-term scalability. This made OpenAI migration a key part of the project, rather than simply replacing one model with another.
Mariana.AI also needed to keep its current workflows unchanged, so the evaluation had to run smoothly. For this, Mriana.AI collaborated with GoML to benchmark Claude models on AWS Bedrock against its current OpenAI setup and understand their performance and potential in creating AI clinical notes.
The solution: A modernized AWS stack for AI in clinical notes
GoML collaborated with Mariana.AI to build the proof of concept using the AI Matic platform as the delivery foundation for its clinical documentation and medical coding workflow. The team benchmarked Claude models (Sonnet 3.5, 3.7, and 4) against their existing OpenAI-based system. The assessment checked the quality of different model families, concentrating on tone, accuracy and clinical alignment. This setup remained separate from Mariana.AI’s production backend, so the team could compare the results without compromising live workflows.
This method was centered around four major areas: OpenAI migration, prompt orchestration, model benchmarking, and output validation. Initially, GoML converted medical papers into standardized JSON and Markdown formats, then they utilized Portkey and LangChain to prompt management over AWS Bedrock. Additionally, the developers installed the benchmarks based on accuracy, sensitivity and F-score as well as schema, logging, and error validation to ensure consistency in clinical outcomes.
AWS Bedrock and Langchain at the core of clinical AI orchestration
The modern AI in clinical documentation pipeline used:
- Prompt Orchestration: Powered by Portkey and Langchain. Routed structure prompts Claude Sonnet Family Models.
- Claude Sonnet: The Core LLM supported intelligent clinical report generation and medical coding.
Enterprise-grade architecture for scalable AI in clinical notes
- Preprocessing Pipelines: ECR containers extracted and standardized notes from physician tools.
- Vector DB Indexing: Embeddings were stored in OpenSearch for context-rich retrieval.
- Explainability Framework: A dedicated Chief Medical Officer (CMO) agent signd off on validated reports.
- CI/CD: GitHub, Docker, and CloudFront pipelines supported fast iteration and DevOps readiness.
AI Matic delivery
GoML used the AI Content Generation blueprint as the delivery foundation for Mariana.AI clinical documentation and medical coding workflow.
Content Generation blueprint supported report generation, prompt orchestration, schema validation, benchmarking, and the migration form present OpenAI based system to AWS bedrock.
This approach of delivery kept the POC separate from Mariana.AI production backend. This allows both the companies to benchmark Claude models and evaluate clinical alignment before changing on-going workflows.
AI Matic delivery metrics
- Average TTFV: 20 days
- Average person-days saved: 8 days
Impact
Mariana.AI now benefits from consistent, high-quality clinical reports generated by AI, with explainability, benchmarking, and compliance built-in.
- 82% reduction in manual report verification time due to structured JSON validation and explainability checks.
- 3x faster faster generation of AI clinical notes over legacy systems
- 97% adherence to output schema across specialties like cardiology, neurology, and gastroenterology.
- 65% improvement in response accuracy after benchmarking with real-world inputs using F-score analysis.
About
Before Gen AI and after Gen AI
“With this migration wasn’t just about switching APIs, it was about engineering trust, speed, and structure into every AI-generated report”
Prashanna Rao, Head of Engineering at GoML.
Common challenges
- Strong accuracy, compliance and scalability controls are needed in clinical documentation systems.
- Preparing AI clinical notes requires proper validation to reduce manual review and QA effort.
- Improper outputs increase risk of uneven clinical documentation
- Managing different documents and specialty requirements requires orchestration of prompt workflows.
- AI agents need separation between generation, validation, and explainability.
- Clinical teams need accurate methods for measuring output quality before using AI-generated systems.
Practical guidance
- Begin with continuous clinical documentation tasks that suit AI automation.
- Use schema validation to keep every GenAI output accurate and consistent.
- Apply benchmarking to measure accuracy and documentation quality.
- Use multi-agent frameworks to separate generation, validation, and explainability workflows.
- Add RAG with validation layers to improve the reliability of generated clinical outputs.
- Use automated prompt orchestration and specialty-focused agents to reduce manual editing and clinical QA cycles.
Ready to modernize clinical documentation with AI
Partner with GoML to build accurate, structured, and scalable AI clinical notes and medical coding workflows on AWS with AI Matic.
