Back to report
Signal 05 of 10

Whether AI crawlers can reach your site at all

95/100Strongwww.thecreditpeople.com
95/100
Strong

Whether AI crawlers can reach your site at all

This signal is helping buyers find and trust you.

You vs peer median
Peer median 72

Why this matters

SEO foundation signals determine whether AI crawlers can reach your site at all. Before AI can recommend you, cite you, or sell for you, it has to be able to read your pages. robots.txt, sitemaps, SSL, page speed, and mobile viewport are the gatekeepers.

This is not traditional SEO. AI crawlers do not rank pages in a list of 10 blue links. They parse your site once, extract what they can, and decide whether to include you in their knowledge base. A single misconfigured robots.txt directive can make your entire site invisible to AI.

The gold standard
  • HTTPS with valid SSL certificate
  • robots.txt that allows AI crawlers (Googlebot, GPTBot, ClaudeBot, PerplexityBot)
  • XML sitemap with all indexable pages discoverable
  • Page load time under 2 seconds
  • Mobile viewport tag
  • Exactly 1 H1 tag per page
  • Meta title (30-60 chars) and description (120-160 chars)
  • Canonical tags on every page
  • Zero images without alt text

LLM crawlability score

Page Load Speed3953ms — acceptable. Under 2,000ms preferred for priority crawling.
Server-Side Rendered ContentNo substantial text blocks in raw HTML. AI bots cannot extract meaningful content.
SSL / HTTPSSite secured with HTTPS. Required for all AI agent integrations.
Mobile ViewportMobile viewport declared. AI crawlers use mobile-first indexing.
Rich Schema MarkupZero structured data. AI systems have no machine-readable content to extract.
Answer-First Content StructureNo answer-block paragraphs detected. AI systems cannot extract direct answers — they skip to sites that have them.
Preload / Performance HintsNo preload hints found. Adding <link rel="preload"> improves perceived load time for all bots.

What we found on your site

HTTPS / SSLSite secured with HTTPS
Title tag (42 chars)42 characters (optimal: 30-60)
Meta description (128 chars)128 characters (optimal: 120-160)
Canonical tagCanonical URL declared
robots.txtFound
XML sitemapFound (11325 entries)
Mobile viewportMobile-first indexing supported
H1 tagSingle H1 tag
Page load time (3953ms)Acceptable but under 2s preferred.
Image alt text21 image(s) missing alt text

Your robots.txt

#
# robots.txt
#
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.
#
# This file will be ignored unless it is at the root of your host:
# Used:    http://example.com/robots.txt
# Ignored: http://example.com/site/robots.txt
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/robotstxt.html

User-agent: *
# CSS, JS, Images
Allow: /core/*.css$
Allow: /core/*.css?
Allow: /core/*.js$
Allow: /core/*.js?
Allow: /core/*.gif
Allow: /core/*.jpg
Allow: /core/*.jpeg
Allow: /core/*.png
Allow: /core/*.svg
Allow: /profiles/*.css$
Allow: /profiles/*.css?
Allow: /profiles/*.js$
Allow: /profiles/*.js?
Allow: /profiles/*.gif
Allow: /profiles/*.jpg
Allow: /profiles/*.jpeg
Allow: /profiles/*.png
Allow: /profiles/*.svg
# Directories
Disallow: /core/
Disallow: /profiles/
# Files
Disallow: /README.txt
Disallow: /web.config
# Paths (clean URLs)
Disallow: /admin/
Disallow: /comment/reply/
Disallow: /filter/tips
Disallow: /node/add/
Disallow: /search/
Disallow: /user/register/
Disallow: /user/password/
Disallow: /user/login/
Disallow: /user/logout/
# Paths (no clean URLs)
Disallow: /index.php/admin/
Disallow: /index.php/comment/reply/
Disallow: /index.php/filter/tips
Disallow: /index.php/node/add/
Disallow: /index.php/search/
Disallow: /index.php/user/password/
Disallow: /index.php/user/register/
Disallow: /index.php/user/login/
Disallow: /index.php/user/logout/
Your robots.txt may be blocking AI crawlers. Check for Disallow directives targeting GPTBot, ClaudeBot, PerplexityBot, or broad Disallow: / rules.

Sitemap at https://www.thecreditpeople.com/sitemap.xml with 11325 entries.

What to do next

    Learn more