Add Your Website for Training
Follow simple steps to add your website to Data Sources, and let Witzo automatically extract and structure your content for accurate AI interactions.
Overview
Website scraping is the primary way Witzo builds your knowledge base. Once added, your pages are crawled, processed, and indexed for AI-powered conversations. This ensures your AI has the most up-to-date information directly from your site.
Add your website
- Go to Dashboard → Agent → Data Sources.
- Go Your Website → Add Website URL.
- Click on Train Agent
- Start training.
How scraping works
Witzo automates website ingestion by crawling pages, extracting readable content, and indexing the results for retrieval. This enables retrieval-augmented generation (RAG), so responses are grounded in your actual website content.
Scraping detection
Witzo checks for website changes before updating scraped data:
- Modified content is detected by comparing page changes.
- Newly added URLs are identified during crawl updates.
Crawl limits
Limits are determined by your active subscription plan.
Important Notes
- Only public content is scraped by default.
- Large sites may take longer to process.
- Authenticated pages are not supported.
- JavaScript-heavy sites may have partial coverage.