Technology
RAG with Google Vertex AI and BigQuery
Development of a rapid Proof of Concept (POC) for a Retrieval-Augmented Generation (RAG) system, integrating Google Vertex AI, BigQuery, and advanced LLM capabilities to enhance data query and document processing.
Overview
Developed a robust RAG system using multiple LLMs, integrated with Google Vertex AI and BigQuery to handle complex queries, enhance LLM responses, and improve document ingestion and retrieval for seamless workspace management.
The solution
Developed a POC for a RAG-based system that allows for LLM integration with Google Vertex AI and BigQuery, enabling flexible query handling, document processing, and workspace management. The solution also included features for API switching between different LLMs, error handling to reduce API calls, and data ingestion for enhanced query accuracy.
Business challenge
The primary business challenge was to develop a scalable, high-performance RAG system that could efficiently handle large datasets and provide accurate LLM responses. This involved creating a seamless integration between Google Vertex AI and BigQuery while maintaining real-time processing capabilities and minimizing operational costs associated with API calls.
Technical challenge
The technical challenges included ensuring smooth integration between multiple platforms (Google Vertex AI, OpenAI, and BigQuery) and managing large volumes of data while maintaining performance. Further challenges arose with the implementation of streaming response mechanisms, database reset functionalities, and adapting the LangChain framework for indexing in BigQuery, which did not have native support for LangChain indices.
What we built
Key features
Generative AI Chat App
Created a chat app using Streamlit and integrated LLM functionalities.
Multi-Column Layout
Implemented a multi-column layout within the chat app for better organization.
Database Reset
Added functionality to reset the database using Chroma DB's reset function.
LLM Switching
Developed an API to toggle between OpenAI and Vertex AI models.
Streaming Response in FastAPI
Integrated streaming response capabilities for better user interaction.
Workspace Management
Added workspace creation, selection, and file upload functionality.
PDF Ingestion
Provided support for PDF uploads and ingestion using GCP BigQuery.
Error Handling
Developed error-handling mechanisms to reduce unnecessary API calls and improve system efficiency.
Chunking Support
Added chunking capabilities to show relevant content during queries.
Delete Dataset and File Functionality
Implemented features to delete datasets and files from GCP storage.