pii · Guide

Detection modes

Four modes, opt-in via detection.mode, each a strict superset of candidates over the one before it. The last one changes what the neural model is for — a judge instead of another vote.

The four modes

Mode	Adds	Dependencies	Sync or async
pattern (default)	Regex + checksum/format validators.	None.	detect/redact/mask/tokenizeText (sync) or their Async counterparts.
pattern+dictionary	person, organization, location via the built-in Turkish gazetteer.	None (a bundled ~5 KB data file).	Async functions only.
pattern+dictionary+ner	The same three categories, generalised past the gazetteer by the bundled 1.45 MB char-CNN — purely additive, one more noisy-OR input.	onnxruntime-node / onnxruntime (the model file ships inside the package either way).	Async functions only.
pattern+ner-verified	Same candidates as above, but the CNN acts as a judge: every regex/dictionary hit is re-scored by how much the model agrees with that exact span, up or down, instead of only ever reinforcing it.	Same as pattern+dictionary+ner.	Async functions only.

pattern+dictionary+ner is purely additive: the CNN's candidates are one more noisy-OR input, so it can reinforce a false positive but never veto one. An invoice ID or a bare 11-digit run can still come out scored as a creditCard/tc_kimlik finding even when nothing else in the sentence is PII, because the regex/checksum layer's own score never gets pulled down.

pattern+ner-verified: the CNN as a judge

This mode reruns fusion with the model in front of the same candidates (L1 pattern findings, L2 dictionary hits, L3 spans decoded with a BIO-constrained Viterbi decoder rather than unconstrained argmax), and changes its role: for every candidate whose category has a CNN counterpart, it asks the model whether the token span it would tag agrees with this exact span, and pulls the score up or down accordingly.

Which categories the judge can score. Only categories with an entry in CATEGORY_TO_CNN_TYPE:

Category	CNN type
phone, tr_phone, de_phone	PHONE
email	EMAIL
tc_kimlik, ssn_us, passport_en, tr_passport, tr_vkn, tr_plaka	ID
iban, tr_iban, creditCard	FIN
address_tr, address_en	LOC
person	PER
organization	ORG
location	LOC

Everything else — url, ipAddress, apiKey, swift, cryptoWallet, birthDate, socialHandle, any custom pattern, and all six categories added most recently (geoCoordinates, username, password, cardCvv, pin, macAddress) — has no CNN counterpart and passes through unjudged, at its pattern-tier score. So does a candidate reconstructed from spelled-out digit words: the model was never trained on spelled numbers, so it has no basis to agree or disagree with one.

The fusion knobs

All seven live under detection.verify and default as shown (identical in both languages):

Knob	Default	What it does
agreeWeight	0.5	When the model agrees with a span (agreement ≥ 0.5), this weight is noisy-ORed in as reinforcement.
floor	0.25	The floor a disagreeing candidate's score is pulled toward, by default — never dropped to zero.
checksumFloor	0.7	A higher floor for a category whose validator checks real arithmetic the CNN cannot learn: tc_kimlik, tr_vkn, iban, tr_iban, creditCard. A phone or SSN "validator" is a format check, not a checksum, and does not get this floor.
contextFloor	0.8	Raises whichever floor applies (never lowers it) when a category keyword sits next to the match (tel, pasaport, plaka, …), so the model cannot overrule a plate or passport number just because the number itself looks unfamiliar to it.
dictionaryFloor	0.5	Floor for a gazetteer hit that disagrees with the model.
nerOnlyWeight	0.6	Weight for a CNN email span with no overlapping pattern candidate at all.
nerOnlyNumberWeight	0.35	Discounted weight for a CNN phone/ID/financial span with no overlapping pattern candidate — the regex front already rejected that number, so it enters at a discount rather than full weight. CNN person/org/location-only spans are added exactly as in pattern+dictionary+ner, at full weight.

Agreement is checked against a >= 0.5: at or above that, the signal is reinforcement (agreeWeight); below it, the score is pulled toward whichever floor applies rather than to zero.

What this mode changes structurally

• This mode's own default minConfidence is 0.4 when the caller sets none — every other mode defaults to 0.
• A Luhn-failed card candidate (13–19 digits, issuer-prefix-shaped, but the checksum fails) is a real candidate only in this mode: it enters at low confidence and needs a nearby card keyword or CNN agreement to clear minConfidence. In every other mode it is simply not a candidate at all.
• Model/runtime unavailable degrades the same way as pattern+dictionary+ner — a note in degraded, never a throw.
• Decoding uses a BIO-constrained Viterbi decoder over the model's transition matrix instead of unconstrained argmax, which removes structurally invalid spans (a lone I-PER with no preceding B-PER, for example) without inventing new true positives.

detection.ner options

Shared by pattern+dictionary+ner and pattern+ner-verified:

Option	Default	What it does
minScore	0.5	Confidence floor on the decoded per-token label.
maxChars	20000	Input is clipped to this length for the NER scan; a degraded note records the clip.
timeoutMs (JS) / timeoutS (Python)	1500 ms / 1.5 s	Soft timeout per inference call.
maxConcurrent	2	In-flight inference calls allowed at once; further calls wait for a slot.
failMode	'open'	'open' degrades to the dictionary tier on failure/timeout; 'closed' re-throws.
decode	'argmax'	'argmax' (used by pattern+dictionary+ner) or 'viterbi' (BIO-constrained, used by pattern+ner-verified).

A single model call covers at most 160 tokens; longer text is scanned in overlapping 32-token windows and stitched into one decode, so an entity straddling what used to be a hard window boundary is no longer split or dropped — at a cost of roughly 24% more tokens processed on long inputs. See Model & benchmarks for the measured effect.

Which mode to choose

Recommended for Turkish production use: pattern+ner-verified. On the current model it has the best identifier precision of any mode (F1 up to 0.934 on the identifier-probe suite, against 0.795 for the pattern tier alone) and the best CC0 recall, at a latency cost of roughly 0.83 ms p50 per call versus 0.74 ms for pattern+dictionary+ner and well under 0.1 ms for the zero-dependency pattern tier alone.

pattern+dictionary+ner remains the right choice only if you specifically want the CNN's candidates to add recall and never subtract it — for example, a review queue where a human looks at every flagged span and a false positive costs a glance, not an automated decision.

Use pattern alone when you only need the fixed-shape categories (email, phone, card, IBAN, national IDs, and so on) and want zero dependencies and the lowest possible latency — this tier runs in well under a millisecond, even under load.

Where to go next

• Categories — every category, its validator, and which ones the judge can score.
• Turkish specifics — how the gazetteer and judge handle Turkish morphology and homonyms.
• Model & benchmarks — architecture, training data, and the full latency/accuracy tables.
• JavaScript API and Python API — the exact detection option shapes.