Problem Statement: I am building a chatbot in Next.js using ai and @ai-sdk/google. The chatbot should stream responses from Gemini (gemini-1.5-flash-8b) to the frontend. I confirmed that the response is streamed in the backend by logging the chunks:
for await (const textPart of result.textStream) {
console.log(textPart);
}
However, when I return the response to the frontend, the streaming effect is lost, and I receive the full response all at once.
What I Have Tried:
Backend (API Route in Next.js):
import { createGoogleGenerativeAI } from "@ai-sdk/google";
import { streamText } from "ai";
import { NextRequest, NextResponse } from "next/server";
const google = createGoogleGenerativeAI({
apiKey: process.env.GOOGLE_GENERATIVE_AI_API_KEY,
});
const model = google("gemini-1.5-flash-8b");
export async function POST(req: NextRequest) {
try {
const { messages } = await req.json();
console.log(messages);
const result = streamText({
model: model,
system: "You are a helpful assistant.",
prompt: messages[0].content,
});
for await (const textPart of result.textStream) {
console.log(textPart); // Confirms streaming works in backend
}
return result.toDataStreamResponse(); // Returns the response
} catch (error) {
console.error(error);
return new NextResponse("Error processing request", { status: 500 });
}
}
Frontend (useChat Hook from ai/react):
"use client";
import React from "react";
import { Input } from "./ui/input";
import { useChat } from "ai/react";
import { Button } from "./ui/button";
import { Send } from "lucide-react";
import MessageList from "./MessageList";
const ChatComponent = () => {
const { input, handleInputChange, handleSubmit, messages } = useChat();
return (
<div className="relative max-h-screen overflow-scroll" id="message-container">
<div className="sticky top-0 inset-x-0 p-2 bg-white h-fit">
<h3 className="text-xl font-bold">Chat</h3>
</div>
<MessageList messages={messages} />
<form onSubmit={handleSubmit} className="sticky bottom-0 inset-x-0 px-2 py-4 bg-white">
<div className="flex">
<Input value={input} onChange={handleInputChange} placeholder="Ask any question" className="w-full" />
<Button className="bg-blue-600 ml-2">
<Send className="h-4 w-4" />
</Button>
</div>
</form>
</div>
);
};
export default ChatComponent; Expected Behavior: I want the frontend to display the response text as it arrives, rather than waiting for the entire response to be received.
Actual Behavior: The backend correctly logs the response as a stream. The frontend receives the response, but only after all of it is processed. The real-time streaming effect is lost. Question: How can I ensure the response is streamed to the frontend in real time using streamText? Do I need to manually handle the stream in the frontend instead of relying on useChat?
Problem Statement: I am building a chatbot in Next.js using ai and @ai-sdk/google. The chatbot should stream responses from Gemini (gemini-1.5-flash-8b) to the frontend. I confirmed that the response is streamed in the backend by logging the chunks:
for await (const textPart of result.textStream) {
console.log(textPart);
}
However, when I return the response to the frontend, the streaming effect is lost, and I receive the full response all at once.
What I Have Tried:
Backend (API Route in Next.js):
import { createGoogleGenerativeAI } from "@ai-sdk/google";
import { streamText } from "ai";
import { NextRequest, NextResponse } from "next/server";
const google = createGoogleGenerativeAI({
apiKey: process.env.GOOGLE_GENERATIVE_AI_API_KEY,
});
const model = google("gemini-1.5-flash-8b");
export async function POST(req: NextRequest) {
try {
const { messages } = await req.json();
console.log(messages);
const result = streamText({
model: model,
system: "You are a helpful assistant.",
prompt: messages[0].content,
});
for await (const textPart of result.textStream) {
console.log(textPart); // Confirms streaming works in backend
}
return result.toDataStreamResponse(); // Returns the response
} catch (error) {
console.error(error);
return new NextResponse("Error processing request", { status: 500 });
}
}
Frontend (useChat Hook from ai/react):
"use client";
import React from "react";
import { Input } from "./ui/input";
import { useChat } from "ai/react";
import { Button } from "./ui/button";
import { Send } from "lucide-react";
import MessageList from "./MessageList";
const ChatComponent = () => {
const { input, handleInputChange, handleSubmit, messages } = useChat();
return (
<div className="relative max-h-screen overflow-scroll" id="message-container">
<div className="sticky top-0 inset-x-0 p-2 bg-white h-fit">
<h3 className="text-xl font-bold">Chat</h3>
</div>
<MessageList messages={messages} />
<form onSubmit={handleSubmit} className="sticky bottom-0 inset-x-0 px-2 py-4 bg-white">
<div className="flex">
<Input value={input} onChange={handleInputChange} placeholder="Ask any question" className="w-full" />
<Button className="bg-blue-600 ml-2">
<Send className="h-4 w-4" />
</Button>
</div>
</form>
</div>
);
};
export default ChatComponent; Expected Behavior: I want the frontend to display the response text as it arrives, rather than waiting for the entire response to be received.
Actual Behavior: The backend correctly logs the response as a stream. The frontend receives the response, but only after all of it is processed. The real-time streaming effect is lost. Question: How can I ensure the response is streamed to the frontend in real time using streamText? Do I need to manually handle the stream in the frontend instead of relying on useChat?
It seems you shouldnt be using useChat(), instead use useCompletion(), more info here. Hope it helps, let me know if it worked