Why Your Site Doesn't Show Up in Search or AI
Tomohiro Iida · Published June 25, 2026 · Updated June 25, 2026
A self-diagnosis checklist for B2B companies, working through crawling, indexing, structured data, and AI readiness to figure out why a site is not showing up in search or AI.
Key takeaways
- Start by splitting "not showing up" into three distinct problems: not appearing in search at all (an indexing problem), appearing but not ranking (a content and evaluation problem), and not appearing in AI (an AI-readiness problem). The cause and the fix differ for each.
- The top priority is fixing indexing show-stoppers: mistaken noindex tags, robots.txt misconfiguration, and canonical errors. If these are not cleared up, later efforts tend not to pay off.
- Some of this you can diagnose yourself from public information; some of it requires real GA4 and Search Console data to judge. This article maps out where that line falls.
First, split "not showing up" into three problems
"Not showing up" covers three fairly different situations. Work out which one you are dealing with first, since the cause and the fix differ (and more than one can be happening at once).
| Symptom | Layer to suspect | Where to check |
|---|---|---|
| (a) Not appearing in search at all | Indexing | A site: search, or the "Pages" report in Search Console |
| (b) Appearing, but not ranking | Content and evaluation | Match to search intent, internal linking, page speed |
| (c) Not appearing in AI | AI readability | robots.txt, structured data, first-party information |
Self-diagnosing "not appearing in search at all"
The most serious case is a page that is not indexed by search engines at all. Work through these checks in order.
- Check whether you are indexed: search "site:yourdomain.com". Nothing coming back suggests the whole site is unindexed; add a path to check a single page.
- Check the "Pages" report in Search Console: it lists non-indexing reasons by count — "Discovered/Crawled - currently not indexed," redirect errors, 404s, server errors (5xx).
- Check for a mistaken noindex or X-Robots-Tag: noindex header. A staging-only noindex surviving into production after a relaunch is a common cause.
- Check robots.txt: confirm an important directory has not been disallowed, which stops crawling and indexing alike.
- Check canonical tags: a canonical pointing to the wrong page on every page can push search engines to treat a different URL as authoritative.
- Check your sitemap: confirm it is submitted and fetched in Search Console; stale URLs or 404s inside it can hurt evaluation.
- Check JavaScript rendering: if body content only renders client-side, crawlers may miss it. Check the page source or the rendered HTML in URL Inspection.
- Check server stability: frequent 5xx errors or timeouts reduce crawl frequency.
If this layer is not cleared, later efforts tend not to pay off. It is the highest-priority area to fix.
Self-diagnosing "appearing, but not ranking"
- Match to search intent: does the page answer what someone searching the target keyword actually wants? Intent broadly splits into informational (what is X, how to), transactional (quote, pricing), navigational (company/service name), and commercial-investigation (comparisons, case studies) — corporate-brochure phrasing alone rarely satisfies the last two.
- Content depth and originality: pages that only restate generic industry points tend to be undervalued; first-party material (procedures, case examples, figures, lessons from failure) demonstrated through the writer's real experience and evidence (experience, expertise, authority, trust) is what stands out.
- Internal linking and topic clusters: check whether related pages actually link to each other. Linking a comprehensive hub page to individual spoke articles, in both directions, conveys expertise; an isolated page attracts little authority.
- Keyword cannibalization: several pages inside the same site targeting one keyword can split authority so all of them underperform. Aim for roughly one page per target query, and merge or restructure overlapping pages.
- Speed and mobile: poor Core Web Vitals (LCP, INP, CLS), layout breakage, or hard-to-tap buttons on mobile put you at a disadvantage.
Self-diagnosing "not appearing in AI" (AIO)
Appearing in AI Overviews, ChatGPT, Perplexity, and similar AI search requires a different lens than human-facing SEO, on top of a solid SEO foundation.
- Check you are not blocking AI crawlers: training crawlers (OpenAI's GPTBot, Anthropic's ClaudeBot) and search-facing crawlers that fetch pages to cite in an answer (OpenAI's OAI-SearchBot, PerplexityBot) are different. Appearing in AI answers requires, at minimum, not blocking the search-facing ones in robots.txt — unintentional blocking is not uncommon. Google's "Google-Extended" is not a separate crawler; it is a robots.txt token controlling whether already-crawled content trains or grounds Gemini, and blocking it does not affect Google Search rankings. Whether to allow AI training use is a separate policy decision.
- Check whether your facts are machine-readable: company information, services, pricing, and FAQs written as structured text (not only inside images or a PDF), organized with headings and bullet lists.
- Write in a citable way, at the passage level: AI often cites a single paragraph rather than a whole page. Passages that get cited tend to lead with the conclusion, cover one point per paragraph, and state figures, conditions, or sources concretely.
- Entities and third-party mentions: AI and search engines try to recognize a company as a distinct entity. Consistent name, address, and contact details (NAP) on your own site, plus mentions on third-party sites, help with that recognition — though no specific technique guarantees inclusion.
- Where llms.txt fits: a proposed file format for summarizing a site for AI. It is not an established standard referenced by the major AI search engines, but it costs little to prepare and is worth having on your radar.
Structured data essentials
Structured data is a machine-readable annotation for conveying your site's content accurately to search engines and AI, typically written as JSON-LD. For a B2B site, the main types worth knowing are:
- Organization: company name, logo, address, contact details — helps machines understand your company information.
- WebSite: basic information about the site as a whole.
- BreadcrumbList: conveys your page hierarchy correctly.
- Article / BlogPosting: author, publish date, and headings for articles.
- FAQPage: not required. Write your FAQ clearly as body text first, and treat structured data as a supplementary layer; adding it does not guarantee rich results in search or citation in an AI answer.
Structured data is a support for conveying your content accurately; it does not improve the content itself. Structured data that does not match reality is counterproductive.
A fix-it roadmap, in priority order
You do not need to fix everything at once. Splitting the work into stages by impact and effort is more efficient. Investing in the third stage while the foundation is still broken tends not to pay off.
| Stage | Main work | Goal |
|---|---|---|
| Stage 1 (immediate, critical) | Fix mistaken noindex, robots.txt blocking, canonical errors, and 5xx errors | Recover indexing |
| Stage 2 (foundation) | Titles/meta, structured data, internal linking, sitemap upkeep | Build the basis for conveying content correctly |
| Stage 3 (compounding) | Content depth, AI-oriented writing, ongoing measurement and improvement | Grow evaluation and citation |
Set up measurement — always check the effect after a fix
- Search Console: impressions, clicks, average position, and CTR by query, plus indexing status.
- GA4: sessions by traffic channel, and outcomes (such as inquiry conversions) by landing page.
- Traffic from AI: whether AI services show up as a referral source (see our related article on how much traffic actually comes from AI search).
High impressions but few clicks points to a title or meta description worth improving; clicks with no resulting inquiries points to content or the call to action. Work backward from the numbers to decide what to fix.
Where self-diagnosis reaches its limit
Everything above can largely be triaged from public information. Beyond this point, judgment gets harder without real data: which keywords are losing you traffic, the real state of rankings and CTR per query, and where to prioritize limited time all require reading actual GA4 and Search Console data rather than a general impression. That is where self-diagnosis ends and a professional diagnosis begins.