Core Concepts

Understanding these core concepts will help you get the most out of QuantSearch.

Sites

A site is a website or web application that you want to make searchable. Each site has its own:

  • Index of content
  • Crawler configuration
  • Public access settings
  • Analytics

Your plan determines how many sites you can have and how many pages per site.

Crawling

Crawling is the process of discovering and downloading pages from your website. Our crawler:

  1. Starts at your base URL
  2. Follows internal links to discover pages
  3. Respects robots.txt by default
  4. Handles JavaScript-rendered content (if enabled)

You can configure the crawler to exclude certain paths, limit depth, or only crawl specific URLs.

Content Processing

After crawling, each page goes through content processing:

  1. Cleaning - Remove navigation, headers, footers, and boilerplate
  2. Extraction - Extract the main content, title, and metadata
  3. Summarization - Generate a brief summary for search results
  4. Tagging - Automatically generate relevant tags
  5. Embedding - Create vector embeddings for semantic search

This AI-powered processing means searches find content by meaning, not just keyword matches.

Embeddings & Semantic Search

Traditional search looks for exact keyword matches. Semantic search understands meaning.

When you search for "how to change my login credentials", semantic search also finds content about:

  • "Reset password"
  • "Update account settings"
  • "Modify user authentication"

This is powered by vector embeddings - numerical representations of text that capture semantic meaning. Similar concepts have similar embeddings.

AI Chat

AI Chat goes beyond search by generating natural language answers. When a user asks a question:

  1. We find the most relevant content using semantic search
  2. We send that content to an LLM (Large Language Model)
  3. The LLM generates a conversational answer
  4. We return the answer with source links

Chat can maintain sessions for multi-turn conversations, remembering context from previous questions.

Public Access

Public access allows unauthenticated search from your website. When enabled:

  • Visitors can search without logging in
  • Requests are validated by origin (CORS)
  • Rate limits protect against abuse

You control which domains can access your search, and whether to enable AI chat or just basic search.

Search Groups

Search Groups (Enterprise only) let you search across multiple sites at once. For example:

  • Search docs.example.com and support.example.com together
  • Combine multiple regional sites
  • Aggregate content from multiple products

Results include which site each result came from, and you can configure separate rate limits for the group.

API Access

API access (Enterprise only) gives you programmatic control over QuantSearch:

  • Ingest content directly without crawling
  • Build custom search interfaces
  • Integrate with your CMS or backend
  • Automate index management

API keys are managed in your dashboard and can be revoked at any time.