Data Methodology

How Forelight works, where the data comes from, what the numbers are, and what we don't track. No black boxes.

<5 min

Median detection lag (cert issuance → API)

<2%

Domain detection false positive rate

CT logs monitored across 4 operators

Person-level PII fields in schema

The data source: Certificate Transparency logs

Under RFC 6962 and its successor RFC 9162, every publicly trusted Certificate Authority is legally required to submit every SSL/TLS certificate it issues to public Certificate Transparency (CT) logs before the certificate is considered valid. This is a browser-enforced requirement — Chrome and Safari will reject certificates not in the CT logs.

This means every time any company anywhere on the internet gets an SSL certificate for a new domain or subdomain, a public record of that event appears in a CT log within minutes. Forelight reads those logs continuously in real time.

Legal basis: CT logs are public infrastructure mandated by RFC 6962. Forelight runs its own log follower against public CT log APIs — no scraping, no terms of service violations, no intermediaries.

Ingestion pipeline

Step 1 — Log following

We follow the CT log tree using the standard get-entries API. Each log operator (Google, Cloudflare, Let's Encrypt, DigiCert, Sectigo) exposes a public HTTP endpoint. We poll continuously and checkpoint our position in the tree so we never miss an entry or double-process one.

Step 2 — Certificate parsing

Each log entry contains a DER-encoded certificate or pre-certificate. We parse it using the cryptography library and extract: Subject Alternative Names (SANs), subject CN, issuer, validity window, and the TBS (to-be-signed) hash. We store the TBS hash as our primary key — never the raw certificate blob.

Step 3 — Deduplication

We deduplicate on TBS hash, not leaf hash. A pre-certificate and its corresponding final certificate share a TBS hash but have different leaf hashes. Deduplicating on leaf hash would double-count every issuance. Deduplicating on TBS hash correctly treats each issuance as a single event.

Step 4 — Enrichment

For each new domain we detect, we run:

DNS resolution — A/AAAA records via async resolver
ASN + hosting fingerprint — IP-to-ASN mapping to identify AWS, Cloudflare, GCP, etc.
SaaS vendor detection — SAN pattern matching against 40+ vendor patterns (Shopify, HubSpot, Salesforce, Vercel, etc.)
Firmographic enrichment — Company name and industry via People Data Labs Company API (company-level only, no person data)

Step 5 — Signal classification

Enriched domain records are classified into typed signal events. Classification rules are deterministic — a signal either matches or it doesn't. No scoring model is applied at this stage.

CT logs monitored

Log	Operator	Status
nimbus2025	Cloudflare	Active
xenon2025h2, xenon2026h1, argon2026h1	Google	Expansion planned
oak2025h2, oak2026	Let's Encrypt	Expansion planned
yeti2025h2, yeti2026h1, nessie2026h1	DigiCert	Expansion planned
sabre2025h2, mammoth2026h1	Sectigo	Expansion planned

Each new CT log starts from the current tree head when enabled — it does not backfill historical entries.

Detection latency

Our p50 detection latency (cert issuance → signal available in API) is under 5 minutes. The breakdown:

CT log propagation: typically 1-3 minutes after CA submission
Our log follower poll interval: ~30 seconds
Parsing + dedup: <1 second per entry
Enrichment (DNS, ASN, SaaS vendor): 2-10 seconds per domain
Signal classification + write: <1 second

Compare this to intent data vendors, which typically operate on 24-48 hour batch cycles, or web crawlers that update on weekly schedules.

Signal types and detection logic

Signal type	Detection logic	False positive rate
new_apex_domain	First time a root domain (e.g. acme.com) appears in any CT log we monitor	<1%
new_subdomain	First time a subdomain under a known apex appears in any CT log	<1%
saas_adoption_detected	SAN matches a known SaaS vendor pattern (e.g. .myshopify.com, .hubspotpagebuilder.com)	<3%
geographic_expansion	New apex domain with a country-code TLD (60+ markets)	<2%
wildcard_cert_issued	Certificate SAN contains *.domain format	<1%
infrastructure_expansion	5+ new subdomains under same apex in 24 hours	<5%
domain_velocity	3+ new_apex_domain signals for same company in 7 days	~8%*

*domain_velocity false positives are typically large companies with active domain portfolios, not high-growth signals. Use in combination with company size filters for best precision.

Signal scoring

Every signal includes a score (1-100) and a score_reason field that explains what's driving it. Scores are computed from two components:

Base weight — determined by signal type, reflecting historical buying-signal correlation
Recency boost — +10 if detected in the last 24 hours, +5 if in the last 7 days. Signals lose their recency boost as they age but the base score persists.

Scores are not a probability model. They are a fixed weighting of signal type plus a time decay on the recency component. You can filter, sort, and route on score directly in Clay or your own pipeline.

What we don't track

Zero person-level PII. Our schema contains no WHOIS registrant fields, no registrant names, no registrant emails. Firmographic enrichment is company-level only (People Data Labs Company API). We explicitly do not use person-focused enrichment APIs.

No WHOIS registrant data
No individual contact information
No tracking pixels or behavioral web data
No raw certificate blobs (TBS hash + leaf index only)
Phishing-adjacent signal types are gated behind explicit buyer verification

Validate the data yourself

We encourage it. Take a list of 20 domains from your ICP. Call GET /v1/signals?apex_domain=<domain> for each. Cross-reference the detected events against what you know happened at those companies. If we're wrong, tell us — hello@forelight.net.

We also publish our signal detection rules openly and will share sample datasets for evaluation. Ask.

Questions?

Email hello@forelight.net or read the full API docs.