Core Concepts

Understanding these core concepts will help you get the most out of QuantSearch.

Sites

A site is a website or web application that you want to make searchable. Each site has its own:

  • Index of content
  • Crawler configuration
  • Public access settings
  • Analytics

Your plan determines how many sites you can have, and how many pages you can index in total across all of them. There is no per-site page cap — a single site can use your whole allowance.

Crawling

Crawling is the process of discovering and downloading pages from your website. Our crawler:

  1. Starts at your base URL
  2. Follows internal links to discover pages
  3. Respects robots.txt by default
  4. Handles JavaScript-rendered content (if enabled)

You can configure the crawler to exclude certain paths, limit depth, or only crawl specific URLs.

Page Updates

A page update is one page processed by a crawl or pushed through the API. It is the unit your plan's monthly allowance is measured in, and it is what indexing consumes.

A page update is deliberately not the same as a page indexed. A page the model rejects — a 404, a login wall — still costs a full processing call even though nothing is added to your index, so it counts. A page skipped before processing (too little content) costs nothing and does not count. Crawling the same page again next week counts again, which is why this is a monthly allowance rather than a property of your index.

Two limits therefore work together: pages indexed bounds how large your index can be, and page updates bounds how much work you do on it each month.

When the allowance is used

Indexing pauses. Searches and AI answers on your site keep working — a billing limit never takes your search box down. New crawls are declined until the allowance resets at the start of your next billing period.

We email you at 80%, at 90% and when the allowance is used, so it is not a surprise. To keep indexing past the allowance, turn on overage in billing settings. Overage is off unless you turn it on, and turning it on requires setting a monthly cap — when the cap is reached we stop crawling rather than keep charging. Searches are never billed.

Content Processing

After crawling, each page goes through content processing:

  1. Cleaning - Remove navigation, headers, footers, and boilerplate
  2. Extraction - Extract the main content, title, and metadata
  3. Summarization - Generate a brief summary for search results
  4. Tagging - Automatically generate relevant tags
  5. Embedding - Create vector embeddings for semantic search

This AI-powered processing means searches find content by meaning, not just keyword matches.

Embeddings & Semantic Search

Traditional search looks for exact keyword matches. Semantic search understands meaning.

When you search for "how to change my login credentials", semantic search also finds content about:

  • "Reset password"
  • "Update account settings"
  • "Modify user authentication"

This is powered by vector embeddings - numerical representations of text that capture semantic meaning. Similar concepts have similar embeddings.

AI Chat

AI Chat goes beyond search by generating natural language answers. When a user asks a question:

  1. We find the most relevant content using semantic search
  2. We send that content to an LLM (Large Language Model)
  3. The LLM generates a conversational answer
  4. We return the answer with source links

Chat can maintain sessions for multi-turn conversations, remembering context from previous questions.

Public Access

Public access allows unauthenticated search from your website. When enabled:

  • Visitors can search without logging in
  • Requests are validated by origin (CORS)
  • Rate limits protect against abuse

You control which domains can access your search, and whether to enable AI chat or just basic search.

Search Groups

Search Groups (Enterprise only) let you search across multiple sites at once. For example:

  • Search docs.example.com and support.example.com together
  • Combine multiple regional sites
  • Aggregate content from multiple products

Results include which site each result came from, and you can configure separate rate limits for the group.

API Access

API access (Enterprise only) gives you programmatic control over QuantSearch:

  • Ingest content directly without crawling
  • Build custom search interfaces
  • Integrate with your CMS or backend
  • Automate index management

API keys are managed in your dashboard and can be revoked at any time.