Query Processing Pipeline for Contextual AI Assistant

This document outlines the comprehensive query processing pipeline for the Contextual AI Assistant, detailing how user queries are analyzed, processed, and answered using the Mem0 memory system with Neo4j as the graph store.

1. Query Processing Pipeline Overview

┌──────────────────┐     ┌──────────────────┐     ┌──────────────────┐
│                  │     │                  │     │                  │
│  Query           │     │  Query           │     │  Intent          │
│  Reception       ├────►│  Preprocessing   ├────►│  Classification  │
│                  │     │                  │     │                  │
└──────────────────┘     └──────────────────┘     └─────────┬────────┘
                                                            │
                                                            ▼
┌──────────────────┐     ┌──────────────────┐     ┌──────────────────┐
│                  │     │                  │     │                  │
│  Response        │     │  Context         │     │  Strategy        │
│  Generation      │◄────┤  Integration     │◄────┤  Execution       │
│                  │     │                  │     │                  │
└──────────────────┘     └──────────────────┘     └─────────┬────────┘
        │                                                   │
        │                                                   ▼
        │                                         ┌──────────────────┐
        │                                         │                  │
        │                                         │  Memory          │
        │                                         │  Retrieval       │
        ▼                                         │                  │
┌──────────────────┐                             └──────────────────┘
│                  │
│  Response        │
│  Delivery        │
│                  │
└──────────────────┘

2. Detailed Pipeline Components

2.1 Query Reception

Purpose: Receive and validate incoming user queries

Components:

1. API Endpoint Handler
   • Receives HTTP POST requests to /chat_completion
   • Validates request format and required fields
   • Extracts query text and user identification
2. Request Validation
   • Ensures query meets minimum length requirements
   • Validates user authentication (if applicable)
   • Performs rate limiting and abuse prevention
3. Query Logging
   • Records incoming queries for analysis and improvement
   • Generates unique query ID for tracking
   • Timestamps the request for temporal context

Input Example:

{
  "query": "What messages haven't I replied to on WhatsApp?",
  "user_id": "user_123"
}

Output: Validated query object with metadata

2.2 Query Preprocessing

Purpose: Prepare the raw query for intent classification and processing

Components:

1. Text Normalization
   • Converts to lowercase
   • Removes excess whitespace
   • Handles special characters
2. Entity Recognition
   • Identifies named entities (people, applications, projects)
   • Extracts temporal references (today, yesterday, next week)
   • Detects application names (WhatsApp, GitHub, etc.)
3. Query Expansion
   • Expands abbreviations (PR → Pull Request)
   • Resolves aliases (Alice → Alice Smith)
   • Handles common synonyms (messages/chats)