AI Search: Grounding Learnings

Sign Up for early access to our Sales Intelligence Tool here

The Gemini Search Grounding Reality

If you've been working with Large Language Models like me, you've likely hit the same frustrating wall I did the knowledge cutoff problem. I've spent months trying to get AI assistants to provide real-time information without resorting to clunky, expensive third-party tools. Then Google quietly rolled out search grounding for Gemini through the Vercel AI SDK (and their own documentation), and it's changed my development approach entirely.

After diving deep into the implementation for my sales intelligence app WhatAIdea, I've discovered that Google's offering isn't just another incremental LLM feature it's the most straightforward path to building applications that can reason about current information without ballooning your API costs.

But let's be honest the existing documentation feels incomplete. This implementation guide comes from my own trial and error, not just regurgitating the docs. I'll walk you through the exact steps needed to get a properly grounded Gemini implementation working with the Vercel AI SDK.

Why Google's Approach Outshines Competitors

Having tested numerous grounding solutions over the past year for my sales intelligence tool, I can say with confidence that Google's approach hits a sweet spot that others miss. It's not just that it's Google (though being the original search engine certainly helps) it's how the integration works.

Most RAG implementations require you to create complex vector databases, manage embeddings, and implement search ranking algorithms. With Gemini, that complexity is abstracted away. You simply toggle a parameter, and suddenly your model has access to Google's search infrastructure.

When I tested it against dedicated search APIs like Brave, the results were frankly surprising:

• Laser focused relevance without needing to implement Google dorking techniques
• Search results that actually understand user intent rather than just matching keywords
• Cost efficiency that makes specialized data enrichment APIs look absurdly expensive

For prospecting tools like my WhatAIdea application, this is the difference between burning cash on every query and having a sustainable business model.

Setting Up Your Development Environment

Before diving into the code, you'll need a few prerequisites:

1. A Google AI Studio account to obtain your API key
2. A Next.js project with the Vercel AI SDK installed
3. Basic familiarity with React hooks and server components

Let's start by installing the necessary packages. Open your terminal and run:

npm install @ai-sdk/google @ai-sdk/react ai

Next, you'll need to add your API key to your environment variables. Create or update your .env.local file:

GOOGLE_GENERATIVE_AI_API_KEY=your_api_key_here

The API key is obtained from the Google AI Studio console. If you haven't already, you'll need to set up a project there first. And no - I'm not going to share mine in this blog post. Some things never change, do they?

The Critical Server-Side Implementation

Here's where most tutorials fall short. The server-side implementation of Gemini's search grounding requires specific configuration parameters that aren't well documented. In your Next.js project, create a route handler at app/api/aichat/route.ts:

import { createGoogleGenerativeAI } from "@ai-sdk/google";
import { streamText } from "ai";
import { NextResponse } from "next/server";

export async function POST(req: Request) {
  try {
    const body = await req.json();
    const { messages, settings = { isGroundingEnabled: false, modelId: 'gemini-1.5-pro-latest' } } = body;
    const { isGroundingEnabled, modelId = 'gemini-1.5-pro-latest' } = settings;
    
    // Create the provider with the API key
    const provider = createGoogleGenerativeAI({
      apiKey: process.env.GOOGLE_GENERATIVE_AI_API_KEY || '',
    });
    
    // Configure model options based on settings
    const modelOptions = {
      useSearchGrounding: isGroundingEnabled,
    };
    
    if (isGroundingEnabled) {
      // Force search retrieval rather than letting model decide
      Object.assign(modelOptions, {
        dynamicRetrievalConfig: {
          mode: 'MODE_UNSPECIFIED' // Always use search when grounding is enabled
        }
      });
    }

    // Create the model instance with options
    const model = provider(modelId, modelOptions);
    
    // Define your system prompt
    const systemPrompt = `You are an AI assistant that helps users with information.`;
    
    // Configure the streamText options
    const streamOptions = {
      model,
      temperature: 0.7,
      maxTokens: 1024,
      messages: [
        {
          role: "system",
          content: systemPrompt
        },
        ...messages
      ]
    };
    
    // Create the text stream
    const result = streamText(streamOptions);
    
    // Return the data stream response with sources
    const responseOptions = {
      sendSources: isGroundingEnabled, // Only send sources when grounding is active
    };

    return result.toDataStreamResponse(responseOptions);
    
  } catch (error: any) {
    console.error("AI Chat: Error in request processing", { error: error.message });
    
    return new Response(JSON.stringify({ 
      error: error.message || "An unknown error occurred"
    }), {
      status: 500,
      headers: { 'Content-Type': 'application/json' }
    });
  }
}

The trickiest part here is understanding the dynamicRetrievalConfig settings. In my testing, using MODE_UNSPECIFIED works best for forcing the model to always use search when grounding is enabled, rather than having the model decide when to search.

Client-Side Component: The User Interface

Now let's implement the client-side component. Create a file at app/components/AIChat.tsx:

"use client";

import { useState } from "react";
import { useChat } from "@ai-sdk/react";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { Switch } from "@/components/ui/switch";
import { Label } from "@/components/ui/label";

export default function AIChat() {
  // Settings State
  const [isGroundingEnabled, setIsGroundingEnabled] = useState(true);
  const [selectedModelId, setSelectedModelId] = useState('gemini-1.5-pro-latest');
  
  // Initialize chat with settings
  const { messages, input, handleInputChange, handleSubmit, isLoading } = useChat({
    api: "/api/aichat",
    body: {
      settings: {
        isGroundingEnabled,
        modelId: selectedModelId
      }
    },
    initialMessages: [
      {
        id: "welcome-message",
        role: "assistant",
        content: "Hello! I'm your AI assistant. How can I help you today?",
      },
    ],
  });

  // Helper to extract sources from message parts
  const extractSources = (message) => {
    if (!message || !message.parts) return [];
    return message.parts.filter((part) => part.type === 'source');
  };

  // Helper to render sources for a message
  const renderSources = (message) => {
    const sources = extractSources(message);
    if (!sources || sources.length === 0) return null;
    
    return (
      

        
Sources: {sources.length}
        

          {sources.map((part, index) => (
            
              {part.source?.title || new URL(part.source?.url).hostname}
            
          ))}
        
      
    );
  };

  return (
    

      

        Web Search
        
        
          {isGroundingEnabled ? "Enabled" : "Disabled"}
        
      
      
      

        {messages.map((message) => (
          

            
{message.content}
            {message.role === 'assistant' && renderSources(message)}
          
        ))}
        
        {isLoading && (
          

            

              

              

              

            
          
        )}
      
      
      

        
        
          {isLoading ? "Thinking..." : "Send"}
        
      
    
  );

The most important part of the client implementation is the extractSources and renderSources functions. These handle the source attribution that Gemini provides, giving your users transparency about where information is coming from.

I've found that users trust AI responses significantly more when they can see that the information is grounded in reputable sources. It's not just about accuracy - it's about user confidence.

The Integration Lessons I Learned the Hard Way

During my implementation of this feature for WhatAIdea, I hit several roadblocks that weren't obvious from the documentation:

1. Performance optimisation is crucial. Grounded responses take longer than standard responses. Adding visual feedback that the AI is "searching" rather than just "thinking" helps manage user expectations.
2. Not all models are created equal. In my testing, gemini-1.5-pro-latest delivered the best balance of accuracy and response time for search grounding.
3. Source parsing requires defensive coding. The structure of source objects can vary, and your UI needs to handle these variations gracefully.
4. Toggle control is essential. Some queries don't need grounding and will be faster without it. Let users decide.

The most significant insight? Grounding is much more than just a technical feature - it fundamentally changes how users interact with your AI. They start to treat it less like a chat toy and more like a serious research assistant.

For my sales intelligence application, this was transformative. Users went from questioning every response to sharing the results directly with prospects. The source attribution creates immediate credibility.

Strategic Takeaways

1. Cost efficiency - Gemini's search grounding eliminates the need for expensive third-party data enrichment APIs
2. Implementation simplicity - Setting up grounding requires minimal code compared to custom RAG solutions
3. Source transparency - Always display sources to build user trust and credibility
4. User control - Providing a toggle for grounding gives users flexibility for different query types
5. Response structure - Structure your UI to handle the additional complexity of sourced responses

Case Study: WhatAIdea Sales Intelligence

When implementing Gemini grounding for my sales prospecting tool, we saw:

• Significant reduction in the cost of company data enrichment
• Notable increase in user-reported accuracy of results
• Significantly higher user retention due to source transparency

The implementation allowed sales teams to research prospects in seconds rather than minutes, with confidence in the information's accuracy.

Based on research and implementation for WhatAIdea sales intelligence tool. Google Gemini AI documentation can be found at the Vercel AI SDK website.