Stop Blaming the LLM: Why Your AI Architecture is Actually Failing Risk Reviews

🔍 Deep Dive: The Architecture Gap

On FinanceBench, a benchmark for long financial filings:

• Vector RAG: ~50% Accuracy.
• PageIndex: 98.7% Accuracy (reported by VectifyAI).

This gap exists because Vector RAG struggles with "distributed" information. In a drug monograph or legal contract, the answer might be split across a Header, a Table, and a Footnote. Vector RAG retrieves one piece of that data and hallucinates the rest. PageIndex understands that to answer the question, the system must traverse the node graph to read the specific node containing the interaction table securely.

3. A Seven-Question Decision Framework

Before you write a single line of ingestion code, answer these seven questions.

1. Document Structure: Are your docs long (50+ pages) and are they highly structured (chunks with consistent headers)?
2. Cross-Section Reasoning: Does the answer require reading Chapter 2, then skipping to Chapter 5, and then Chapter 9?
3. Auditability: Is the output subject to auditors or regulator compliance? Can you defend the answer? (Vector similarity scores are not admissible; node paths are).
4. Vocabulary Ambiguity: Is there a risk of the user asking in one way ("risk factors") while the document uses another ("procurement traps")?
5. Throughput: Do you need to answer 1,000 queries per second, or 10 per minute?
6. Data Types: Is your data a mix of unstructured emails and structured contracts?
7. Hybrid Potential: Are you willing to pay the orchestration cost to combine both?

🏗️ System Design: Implementing PageIndex

If you determine your architecture requires PageIndex, your system design shifts from a "Map" to a "Tree."

Data Schema Concept: Instead of storing chunks as flat tensors, store parent-child relationships.

• Root Node: The full document.
• Level 1: Main Sections (TOC).
• Level 2: Sub-sections (Headers).

Ingestion Flow:

1. Layout Analysis: Use a form of HTML parsing or layout detection to identify headers.
2. Tree Construction: Build a JSON-LD structure representing the hierarchy.
3. Storage: Store the nodes in a database with pointers (neighboring parent/child pointers).

Query Runtime (Pseudo-Code):

def query_risk(document_tree, question):
    # Step 1: Summarize Table of Contents
    toc_summary = summarize_node(document_tree.toc)
    relevant_section_id = choose_section_based_on_question(toc_summary, question)
    
    # Step 2: Descend the tree (Sequential Retrieval)
    current_node = document_tree.get_node(relevant_section_id)
    context = ""
    
    while current_node.has_children():
        # Decide whether to go deeper into sub-sections
        decision = is_sub_section_relevant(current_node.text, question)
        if decision:
            current_node = current_node.children[0] # Go deeper
        else:
            break
    
    final_answer = construct_answer(current_node.text, question)
    return final_answer, [relevant_section_id] # Trace is built-in

Note the difference: Vector RAG launches 10 parallel requests. PageIndex launches 3 sequential requests. This chain of calls is the only thing that guarantees you know exactly where the answer came from.

⚔️ Comparison: Vector RAG vs. PageIndex

🧑‍💻 Practical Value: Which to Build?

The Decision Matrix

• Build PageIndex if: You are in FinTech, LegalTech, or Pharma. Your documents are PDFs with clear chapters. Action: Set up a layout analysis pipeline for your PDF ingestion.
• Stick with Vector RAG if: You are building a consumer chatbot, FAQ bot, or news aggregator. Action: Focus on optimizing your chunk size and embedding model to reduce hallucinations.
• Build a Router if: You have a mixed corpus. Action: Use a lightweight classifier (NLI model) to route the query to the appropriate backend before it hits the expensive reasoning loop.

Common Pitfalls

• Chunking Inertia: Don't force PageIndex logic onto 1MB of raw text scraped from Reddit. You will incur massive costs for zero gain.
• Ignoring the "Footnote" Problem: The original prompt highlights a critical failure mode. If your answer is in a footnote (often low-ranked in embeddings), PageIndex remains superior.

⚡ Key Takeaways

1. Architecture is Governance first: In regulated industries, retrieval isn't an engineering edge case; it's a trust layer.
2. Meaning is Structural or Neighborhoodal: You cannot retrofit structure onto noise-nature text.
3. Vector RAG is the default trap: It is the "cheap" answer that costs you money in audits and legal disputes.
4. Hybrid is the truth: Most production systems need a routing layer.

❓ FAQ

Q: Can I use both Vector DB and PageIndex? A: Yes. Use Vector RAG to find which documents might have the answer, and use PageIndex to find the exact clause within that document.

Q: Is PageIndex too slow for production? A: Only if your query volume is massive (millions/day). For internal enterprise systems handling thousands of queries, the latency is acceptable.

Q: How do I handle messy documents? A: PageIndex struggles with genuinely messy data. If your headers are inconsistent, Vector RAG is likely the better tool, or you need a pre-processing step to "standardize" the document PDF.

🔗 Related Topics

• Building Robust RAG Pipelines: A System Design Perspective
• Why Vector Databases Are Failing in Banking Applications
• The Governance Playbook for Generative AI in the Public Sector

🎯 Conclusion

The next time an AI fails a "risk review," do not point to the hallucination. Look at your retrieval architecture. If your documents rely on structure rather than proximity, you are likely using the wrong system. Stop blaming the model, and start blaming the architecture.