Paginated pages - those using rel="next" and rel="prev" tags - split content across multiple URLs. While pagination is necessary for large content sets, it creates SEO challenges: search engines must understand the relationship between pages, and link equity gets divided across the series. Poorly handled pagination can lead to duplicate content issues and wasted crawl budget.

This guide shows you how to find every paginated page on your website using the crawler.sh CLI.

Step 1: Install crawler.sh CLI

Install the CLI with a single command:

curl -fsSL https://install.crawler.sh | sh

This downloads the correct binary for your operating system and architecture, places it in ~/.crawler/bin/, and adds it to your PATH. Restart your terminal or run source ~/.bashrc (or ~/.zshrc) to pick up the new PATH entry.

Verify the installation:

crawler --version

Step 2: Crawl the target website

Run a full crawl of the website you want to audit:

crawler crawl https://example.com

The crawler detects rel="next" and rel="prev" link elements on every page. Results are saved as an NDJSON file (.crawl) in the current directory. For larger sites:

crawler crawl https://example.com --max-pages 5000

Step 3: Run SEO audit

Run the SEO analysis on your crawl data:

crawler seo example-com.crawl

The paginated pages check flags every page that uses rel="next" or rel="prev" pagination signals.

Step 4: Identify paginated pages

Look for the Paginated Pages section in the SEO report. Common types of paginated content:

Blog archive pages (page 1, 2, 3, etc.)
Product listing pages in e-commerce sites
Search result pages
Forum threads split across multiple pages
Category or tag archive pages

Review each paginated series to check that pagination tags are correctly implemented - each page should point to the right next and previous URLs.

Step 5: Fix and re-crawl

Ensure your pagination is SEO-friendly:

Verify rel="next" and rel="prev" point to valid, correct URLs in the sequence
Set canonical tags on paginated pages to self-reference - do not canonical all pages to page 1 (Google recommends against this)
Consider “view all” pages for shorter series where loading all content on one page is practical
Ensure paginated pages are crawlable - do not block them with robots.txt or noindex
Use consistent URL patterns (e.g., ?page=2 or /page/2/)

After fixing, re-crawl to verify:

crawler crawl https://example.com
crawler seo example-com.crawl

Why paginated pages matter for SEO

Pagination directly affects how search engines discover and index your content. If pagination signals are broken, search engines may miss content on deeper pages or treat paginated pages as duplicates. Proper pagination handling ensures your full content library gets indexed while concentrating ranking signals where they matter most. Regular audits catch issues like broken next/prev links, missing pages in the sequence, or incorrect canonical setups.