Next.js• 5 MIN READ
Kirito

Kirito

6/10/2026•5 min read

The Day Next.js Middleware Destroyed My SEO

A simple middleware to handle authentication created a redirect loop that got 15,000 pages de-indexed from Google in 48 hours.

The Day Next.js Middleware Destroyed My SEO

Two months ago, I was feeling proud of myself.

I had built a beautiful Next.js 14 app with:

  • App Router
  • Server Components
  • Middleware for auth
  • Internationalization

Everything was fast. Everything worked.

Then my SEO rankings crashed.

The Setup

My middleware looked like this:

// middleware.ts
import { NextResponse } from 'next/server';
import type { NextRequest } from 'next/server';

export function middleware(request: NextRequest) {
  const token = request.cookies.get('auth-token');
  const isAuthPage = request.nextUrl.pathname.startsWith('/login');
  const isProtectedPage = !isAuthPage && request.nextUrl.pathname !== '/';

  // Redirect unauthenticated users to login
  if (isProtectedPage && !token) {
    const loginUrl = new URL('/login', request.url);
    loginUrl.searchParams.set('from', request.nextUrl.pathname);
    return NextResponse.redirect(loginUrl);
  }

  // Redirect authenticated users away from login
  if (isAuthPage && token) {
    return NextResponse.redirect(new URL('/dashboard', request.url));
  }

  return NextResponse.next();
}

export const config = {
  matcher: ['/((?!api|_next/static|_next/image|favicon.ico).*)'],
};

Seems reasonable, right?

The Silent Killer

What I didn't realize: Googlebot doesn't send cookies.

When Googlebot crawled my site:

  1. Request to /blog/awesome-post
  2. Middleware sees token = undefined
  3. Redirects to /login?from=/blog/awesome-post
  4. Googlebot follows redirect
  5. Login page (which I accidentally set to noindex for security)

Every. Single. Page.

Every page redirected to a noindex login page.

Google's crawler saw:

  • 15,000 pages returning 302 redirects
  • All redirects leading to a page with <meta name="robots" content="noindex">

Within 48 hours, Google started de-indexing everything.

The Realization

I first noticed something was wrong when I searched site:mywebsite.com:

No results found for site:mywebsite.com

I almost had a heart attack.

Checking Google Search Console showed:

  • Indexed pages: 3 (only the homepage, somehow)
  • Crawl stats: 15,000 pages with "Redirect" status
  • Coverage report: "Page with redirect" × 15,000

The Fix

I learned that Next.js middleware runs on EVERY request — including from bots.

The correct approach: Don't redirect bots.

// FIXED middleware.ts
export function middleware(request: NextRequest) {
  // Detect Googlebot (and other crawlers)
  const userAgent = request.headers.get('user-agent') || '';
  const isBot = /bot|crawler|spider|googlebot|bingbot|slurp|duckduckbot/i.test(userAgent);
  
  const token = request.cookies.get('auth-token');
  const isAuthPage = request.nextUrl.pathname.startsWith('/login');
  const isProtectedPage = !isAuthPage && request.nextUrl.pathname !== '/';

  // NEVER redirect bots
  if (isBot) {
    return NextResponse.next();
  }

  // Only redirect real users
  if (isProtectedPage && !token) {
    const loginUrl = new URL('/login', request.url);
    loginUrl.searchParams.set('from', request.nextUrl.pathname);
    return NextResponse.redirect(loginUrl);
  }

  if (isAuthPage && token) {
    return NextResponse.redirect(new URL('/dashboard', request.url));
  }

  return NextResponse.next();
}

But wait — there's more.

The Second Problem

Even after fixing the redirect, my pages weren't being re-indexed.

Google had already marked them as "soft 404" because they kept redirecting.

I had to:

1. Request re-crawling

# Using Google Search Console API
curl -X POST "https://indexing.googleapis.com/v3/urlNotifications:publish"   -H "Authorization: Bearer $ACCESS_TOKEN"   -H "Content-Type: application/json"   -d '{
    "url": "https://mywebsite.com/sitemap.xml",
    "type": "URL_UPDATED"
  }'

2. Add proper status codes

// For bot requests, return 200 always
if (isBot) {
  const response = NextResponse.next();
  response.status = 200;
  return response;
}

3. Implemented proper sitemap

// app/sitemap.ts
export default async function sitemap() {
  const baseUrl = 'https://mywebsite.com';
  
  // Fetch all public routes
  const posts = await getPublicPosts(); // Only returns non-auth-required posts
  
  return [
    {
      url: baseUrl,
      lastModified: new Date(),
      changeFrequency: 'daily',
      priority: 1,
    },
    ...posts.map(post => ({
      url: `{baseUrl}/blog/{post.slug}`,
      lastModified: post.updatedAt,
      changeFrequency: 'weekly',
      priority: 0.8,
    })),
  ];
}

The Recovery

It took 3 weeks for Google to re-index everything.

Traffic dropped by 85% in the first week.

Revenue? Down 60%.

All because I forgot that bots don't have cookies.

What I Learned

1. Always Test with Bot User Agents

# Simulate Googlebot
curl -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"   https://mywebsite.com/blog/post

2. Middleware Should Have Bot Exceptions

Unless you're building a fully private app, bots should see public content.

3. Use robots.txt for Auth Routes

# public/robots.txt
User-agent: *
Disallow: /login
Disallow: /dashboard
Disallow: /settings
Allow: /

4. Monitor Search Console Daily

I check Google Search Console every morning now. It's the first sign of SEO problems.

Better Middleware Pattern

Here's my production-ready middleware now:

import { NextResponse } from 'next/server';
import { isBot, isPublicPath, shouldRedirectToLogin } from './lib/seo';

export function middleware(request: NextRequest) {
  const userAgent = request.headers.get('user-agent') || '';
  const botCheck = isBot(userAgent);
  const pathname = request.nextUrl.pathname;
  
  // Bots: Let them see everything (except explicit 404s)
  if (botCheck) {
    // But don't let them index auth pages
    if (pathname.startsWith('/login') || pathname.startsWith('/dashboard')) {
      const response = NextResponse.next();
      response.headers.set('X-Robots-Tag', 'noindex');
      return response;
    }
    return NextResponse.next();
  }
  
  // Humans: Apply auth logic
  const token = request.cookies.get('auth-token');
  
  if (shouldRedirectToLogin(pathname, !!token)) {
    const loginUrl = new URL('/login', request.url);
    loginUrl.searchParams.set('from', pathname);
    return NextResponse.redirect(loginUrl);
  }
  
  return NextResponse.next();
}

Commands to Monitor SEO Health

Check for unwanted redirects

# Using curl with follow redirects
curl -Ls -o /dev/null -w "%{url_effective}
" https://mywebsite.com/blog/post

Check index status via Google API

curl "https://indexing.googleapis.com/v3/urlNotifications/metadata?url=https://mywebsite.com/blog/post"   -H "Authorization: Bearer $ACCESS_TOKEN"

Generate a sitemap index

npx next-sitemap --config next-sitemap.config.js

Conclusion

Next.js middleware is powerful. But it runs in the edge runtime, on every request, including from bots.

Always ask: "What does Googlebot see?"

If the answer is "a redirect to login," you have a problem.

That mistake cost me 3 weeks of SEO recovery and $15,000 in lost revenue.

Now I have a checklist before deploying any middleware changes.

And I never forget that bots don't log in.