A run dies halfway, and the cleanup step deletes everything it did not reach.
A run that was interrupted, partial, or failed on half its items or more removes nothing. Only a run that finished well may take documents out.
Open source · Self-hosted · AEM · Solr · Elasticsearch
Your content in your search index. And kept there, correct.
Viglet Dumont connects AEM, Edge Delivery Services, databases, file shares, web sites and JSON APIs to Turing ES, Solr or Elasticsearch. Then it keeps checking: what changed, what disappeared, which field stopped arriving.
Getting documents into an index is a weekend script. Keeping the index true to the source for years, through outages, template changes and unpublished pages, is the part Dumont is built for.
A failed run never empties the indexDeletions wait for a personSelf-hosted, Apache 2.0
$
docker run -p 30130:30130 ghcr.io/openviglet/dumont-ce:latest- source
- wknd-site (aem · cloud service)
- target
- turing · sn site wknd
- trigger
- cron 0 0 3 * * * · nightly audit
- read
- 4,812 items
- unchanged
- 4,630 skipped by checksum
- indexed
- 171 new or changed
- fragments
- 1 footer XF changed → 9 pages requeued
- rejected
- 1 batch (2 docs) held for next run
- orphans
- 11 → report 2026-10-19, awaiting review
- coverage
- abstract 98.7% (−0.1 since last run)
status ok · nothing was removed without review
Why it exists
Indexes rarely break on day one. They drift.
The four ways a search index goes wrong after launch, and what Dumont does about each one.
A page was unpublished months ago and still turns up in search.
The nightly audit compares the source with the index. Leftovers go into a dated report you apply or discard, and the report expires in a day.
A template change quietly empties a field on four thousand pages.
Field coverage is tracked per source and per run, and drift between runs is reported, so a field that stops arriving shows up as a number.
Every run re-sends the whole site, and the search cluster pays for it.
A SHA-256 checksum per item skips what did not change. Batches the target rejected are resent on the next run instead of being forgotten.
How it fits
Six kinds of source, three engines, one pipeline between them
Every connector runs in the same JVM and goes through the same queue, the same checksums and the same audit. A database and an AEM site are operated the same way.
Sources
Viglet Dumont
Spring Boot 4 · Java 21 · port 30130
- 1 read
- 2 extract
- 3 checksum
- 4 queue
- 5 index
- 6 audit
An embedded, persistent Artemis queue between extraction and indexing, so a slow target backs work up instead of losing it.
Targets
Viglet Turing ES
The default. Dumont sends the field schema before the first document and audits what Turing holds.
Apache Solr
Writes straight to a Solr collection.
Elasticsearch
Writes straight to an index, with optional basic auth.
dumont.indexing.provider = turing | solr | elasticsearch
Connectors
Where your content lives today, read the way it wants to be read
Each connector has its own console screen, its own REST endpoints and an export package, so a source configured once can be moved between environments as a file.
Adobe Experience Manager
On-prem, AMS and AEM as a Cloud Service
Pages, Content Fragments and assets, each read the way its content kind wants. Extraction is declared per content type, so one recipe serves every site that shares the template.
DetailsEdge Delivery Services
aem.live sites, from their query index
Reads the site's query-index.json (and its sitemap, if you want it), takes meta tags and body text by CSS selector, and derives facets from the URL path.
DetailsDatabases (JDBC)
A query, a key, and a typed field map
One SQL query and its primary key become documents. Each column maps to a field with a type, and can be multi-valued, a facet, mandatory or an embedding input.
DetailsWeb crawler
Sites you do not own the CMS of
Start points, allow and deny lists, and CSS selectors per attribute. It reads robots.txt and sitemaps, honours Crawl-delay, and backs off when a host answers 429 or 403.
DetailsFiles and documents
A directory, read with Apache Tika
Walks a directory and extracts the text of each document with Apache Tika, so PDFs, Office files and the rest become searchable alongside the pages that link to them.
DetailsStructured catalogs and JSON APIs
Paged JSON, with request budgets
Ingests catalog JSON that is already normalised, or pages through a JSON API under a daily, weekly or monthly request budget, so a rate-limited source is never overrun.
DetailsAdobe Experience Manager
Search for AEM that reads AEM, not the HTML it renders
On-prem, Managed Services or AEM as a Cloud Service. Content types, fragments and replication events are things the connector understands, not markup it scrapes.
Each content kind, read its own way
Pages through model.json, Content Fragments through the Content Fragment API, assets through Assets Search. A source decides per kind, so nothing is scraped that has an API.
AEM as a Cloud Service without an author login
A source can read from the delivery side instead of the author servlets. On-prem and AMS keep infinity.json and QueryBuilder; Cloud Service needs no author credential at all.
Extraction declared per content type
Composable extractors and versioned recipes, chosen by content type and shared across sites. A new site on the same templates is configuration, not a new connector.
Fragments propagate
When a shared Experience Fragment or Content Fragment changes, every page that embeds it is reindexed. The footer edit reaches search the same day it reaches the site.
Three ways to hear about a change
An Adobe I/O Events webhook, an OSGi bundle on the AEM side that sends replication events, and a journal poller that recovers events the other two dropped.
A wizard for the first source
Test the connection, browse the content tree and pick extractors in the console before the first run, instead of writing a configuration file and hoping.
After the run
An index you can leave running and still trust on Monday
Most connectors stop when the documents are sent. Dumont keeps a record of what it sent, compares it with what the index holds, and tells you when the two disagree.
A failed run cannot empty the index
De-indexing is the last step of a run and it is conditional. An interrupted run, a partial one, or one where at least half the items failed keeps every document where it was.
A nightly audit, against the index itself
On a cron you choose (03:00 by default) Dumont lists what the target holds and compares it with the source. Missing or stale documents are queued again.
Deletions are a report, not a side effect
Documents that left the source but are still indexed go into a dated orphan report. A person applies it or discards it; an untouched report expires after 24 hours.
Coverage and drift, per field
For each source, the share of documents that carry each field, and how that changed since the last run. A field that stops arriving is a number on a screen, not a support ticket.
Field-level provenance
Each document can carry where each of its fields came from, so a wrong value in search leads back to the extractor and the source item that produced it.
Prometheus, and a webhook when it goes wrong
Run outcomes are exposed as dumont_run_* metrics on a separate management port, never the public one. A run that starts warning posts JSON to the webhook you configure.
AI, where it helps
An LLM that drafts the source. You decide what gets indexed.
Dumont uses AI for the slow, fiddly parts: working out the shape of a new source, summarising a coverage report, labelling content. None of it writes to your index without a person or a rule you set.
It drafts, a person confirms
Point source inference at a URL and an LLM drafts the source and its fields. A dry run shows coverage and a diff, and nothing is indexed until someone accepts the draft.
No model inside Dumont
Every AI feature calls the LLM you configured in Turing, with Turing's cache. Dumont holds no provider keys and no prompts of its own to keep current.
Enrichment is opt-in and bounded
LLM classification is off by default. When it is on, it picks from a closed label set, under a daily cap, with a SHA-256 cache so the same text is never paid for twice.
Grounding-safe output
An empty field is left out, never sent as an empty string, and fields map onto Turing's title, abstract and text embedding roles, so a RAG answer cites real content.
Marketplace
One sample per connector, ready to import into Turing
Each sample is a search front end plus the Turing export it needs. Download the package, import it, point the connector at the source, and you have a working search to take apart.
AEM
WKND
Adobe's adventure-travel reference site, indexed from AEM, with a search front end on Turing.
Edge Delivery ServicesWKND EDS
The same site served from aem.live and read from its query index.
FilesCelestial Gallery
A folder of astrophotography (FITS, TIFF and JPEG files) turned into a filterable archive.
DatabaseRetro Arcade
A SQL database of classic arcade games, mapped column by column into a searchable catalogue.
Web crawlerWikipedia Explorer
Wikipedia crawled politely from the Main Page, with Talk, User and File pages filtered out.
The Viglet suite
Dumont fills the index. Turing answers from it.
Dumont works on its own against Solr or Elasticsearch. With Turing it does more: it provisions the field schema before the first document, audits the index, and borrows Turing's LLM for every AI feature.
Viglet Dumont
Moves content into the index
Connectors from AEM, EDS, databases, files, sites and JSON APIs, with audits and a safe finish.
Viglet Turing ES
Search and RAG over it
Semantic Navigation, facets, spotlights and generative answers grounded in what Dumont indexed.
Viglet Shio
An agent-ready CMS
A CMS with a console, an MCP surface for coding agents, files and a delivery API.
Where Dumont is the wrong tool
Written down in the strategy, so nobody has to find out by trying.
A crawler farm
If the job is scraping thousands of hostile sites behind JavaScript, use Scrapy or Playwright. Dumont's crawler is for sites you are allowed to index, read politely.
A warehouse ETL
Airbyte and Fivetran move rows into warehouses. Dumont moves documents into a search index and watches that the index stays true to the source.
AI that publishes on its own
The LLM drafts sources and suggests labels. It never decides on its own what goes into your index or what comes out of it.
Run it
On your infrastructure, next to your content
Connectors need to reach the systems they read, which usually means inside your network. Dumont is one image you run there; there is no hosted tier.
One container, your database
Spring Boot 4 on Java 21, every connector in the same JVM, an embedded queue. H2 out of the box for a trial, PostgreSQL when it matters.
Your sign-in
Local users, groups and roles, Keycloak or an OAuth2 provider for the console; an API key for machine calls. Multi-tenant mode if you run it for several teams.
Observable from day one
Prometheus metrics on the management port (30131), a webhook for warnings, and an optional MongoDB or Redis store for the indexing log.
Try it in one line:
$
docker run -p 30130:30130 ghcr.io/openviglet/dumont-ce:latestOr with Turing beside it:
services:
dumont:
image: ghcr.io/openviglet/dumont-ce:latest
ports:
- 30130:30130
volumes:
- ./store:/app/store
environment:
TURING_URL: http://turing:2700
TURING_APIKEY: <your Turing API key>
turing:
image: ghcr.io/openviglet/turing-ce:latest
ports:
- 2700:2700Then open http://localhost:30130. Standalone jars, a Windows installer and an AppImage are described in the installation guide.
Start here
Read it before you run it
The strategy, the open backlog and every shipped change are in the repository, next to the code.
Documentation
Installation, every connector, configuration reference and the REST API.
Strategy
What Dumont is betting on, and what it deliberately is not trying to be.
Roadmap and changelog
The open backlog and every shipped task, one line each, with its commit.
Source
Apache 2.0, Spring Boot 4 on Java 21, a React 19 console. Issues welcome.
Point it at one source this afternoon.
One container, a console in English or Portuguese, and a dry scan that shows what would be indexed before anything is. Apache 2.0, with the roadmap in the repository.
$
docker run -p 30130:30130 ghcr.io/openviglet/dumont-ce:latest