Progressive Person Matching

Problem Statement

The Registration Portal allows principals (logged-in users) to link other persons to their account for registration purposes - such as family members, team members, or dependents. This creates a sensitive security surface: we must enable legitimate linking while preventing attackers from using the person lookup feature to enumerate personal data.

Security Risks

Risk Description

Data Enumeration

An attacker could probe the system with partial data (e.g., common surnames) to discover if specific individuals exist in the database, violating POPIA principles.

Brute Force Matching

Without rate limiting and minimum proof-of-knowledge requirements, attackers could systematically guess personal details to link unauthorized persons.

Primary Key Exposure

Exposing internal User.id values in API responses enables targeted attacks and leaks implementation details.

Information Disclosure

Returning full personal details for partial matches reveals protected information to unauthorized parties.

POPIA Compliance Requirements

The Protection of Personal Information Act (POPIA) requires:

  • Purpose Limitation - Personal data only used for the stated purpose (registration linking)

  • Security Safeguards - Technical measures to prevent unauthorized access

  • Data Minimisation - Only reveal necessary information

  • Accountability - Audit trail of all access attempts

As-Is: Search-Then-Link Flow

The current implementation uses a two-step approach:

1. User enters search criteria (name, ID, etc.)
2. System returns list of matching persons
3. User selects from results
4. System creates LinkedPerson record

Current Security Gaps

Gap Risk Impact

List-based results

Returns multiple matches with personal data

Enables enumeration of who exists in system

No minimum criteria

Can search with minimal information

Too easy to probe with common data

User.id exposure

Primary key returned in responses

Enables targeted attacks

No progressive disclosure

Full details shown immediately

Violates data minimisation principle

Rate limiting gaps

Per-endpoint only, not per-session

Determined attacker can work around

To-Be: Progressive Matching with Weighted Scoring

Design Overview

The new design replaces "search-then-link" with "type → auto-match → confirm":

1. User types fields progressively
2. Frontend calculates local score (gate before API call)
3. Backend scores candidates, returns ONLY if unique match
4. Masked suggestion shown after delay
5. User confirms → LinkedPerson created with opaque token

Key Security Features

Feature Implementation

Frontend Pre-Scoring Gate

20-point minimum before calling backend. Prevents trivial probing.

Weighted Field Scoring

Different fields contribute different points based on uniqueness value.

Uniqueness Requirement

Backend only returns a match if exactly one candidate meets threshold. Multiple matches return "AMBIGUOUS" requiring more fields.

Delayed Reveal

3-second delay at threshold 50; immediate reveal only at 80+. Gives user time to add more fields for better access rights.

Masked Data Only

Suggestions show deterministically masked name (e.g., J*n****n), age range, gender - never full details until linked. Masking reveals floor(length * 0.25) + 1 characters using a hash-based deterministic selection. User-typed name prefixes that match are also revealed (no new information disclosed).

Opaque Match Tokens

matchToken used during search; LinkedPerson.id returned after linking. User.id never exposed.

Access Level by Score

Score 50-79 → READ (masked view); Score 80+ → READ_WRITE (full access)

Cross-Account Trust

+20 boost if same physical User has verified links in another account (prevents re-verification friction)

Frontend Pre-Scoring (API Gate)

The frontend calculates a local score before calling the backend:

Field Points Rules

First Name

4-7

Sliding scale: 2 chars=4, 3=5, 4=6, 5+=7. Minimum 2 chars.

Last Name

4-7

Sliding scale: 2 chars=4, 3=5, 4=6, 5+=7. Minimum 2 chars.

Date of Birth

5

Full date required. Ignored if full ID provided.

Gender

4

Single selection. Ignored if full ID provided.

ID Number (full)

15

Must be 13 chars AND pass Luhn checksum. Excludes DOB/Gender (no double-dipping).

ID Number (partial)

5

6 digits matching valid YYMMDD (no 13th month). Same value as DOB.

Membership Number

10

Organisation-specific identifier

Email

10

Valid email format

Phone Number

10

Valid phone format

Threshold: 20 points required to call backend or submit form.

Name Sliding Scale Formula

points = min(7, max(4, length + 2))
// 2 chars → 4 pts
// 3 chars → 5 pts
// 4 chars → 6 pts
// 5+ chars → 7 pts

ID Number Validation

Full ID (13 characters):

  1. Exactly 13 digits

  2. Positions 1-6: Valid date (YYMMDD) - no invalid months/days

  3. Position 13: Luhn checksum digit must validate

Double-Dip Prevention: When full valid ID matches, DOB and Gender fields contribute 0 points (information already encoded in ID).

Partial ID (6 digits): Must match valid YYMMDD pattern to score 5 points.

Backend Scoring Weights

person-matching:
  weights:
    # ID Number (authoritative)
    sa-id-number-exact: 60      # Full 13 chars + Luhn; excludes DOB/Gender
    sa-id-number-partial: 25    # First 6 digits (YYMMDD)

    # Date of Birth (ignored if full ID provided)
    date-of-birth-exact: 25

    # Gender (ignored if full ID provided)
    gender-exact: 8

    # Names (sliding scale)
    surname-exact-min: 8        # 2 chars
    surname-exact-max: 15       # 5+ chars
    first-name-exact-min: 8     # 2 chars
    first-name-exact-max: 15    # 5+ chars

    # Contact info
    email-exact: 20
    mobile-exact: 20
    membership-number-exact: 40

  thresholds:
    minimum-to-suggest: 50      # Show masked suggestions
    confident-match: 80         # Highlight as likely match

  cross-account-trust-boost: 20 # Same User verified in other account

Response States

Status Meaning UI Action

NO_MATCH

No candidates meet threshold

Continue to new person creation

AMBIGUOUS

Multiple candidates qualify

Prompt for more fields (show suggestions)

UNIQUE_MATCH

Exactly one candidate

Show masked suggestion

Cross-Account Trust Model

When a user logs in via different authentication methods (e.g., Facebook vs username/password), they create separate OrgUser accounts but are the same physical User.

Trust Resolution:

  • If the same User has a verified link (accessLevel=READ_WRITE) to a person in another account, +20 boost applied

  • Alternatively, if the link has been active for >30 days, trust is assumed

  • After linking 2+ people that match another account’s verified links, existing READ links are upgraded to READ_WRITE

API Endpoints

Progressive Match

POST /api/people/progressive-match
Content-Type: application/json

Request:
{
  "input": {
    "idNumber": "8501015009087",
    "firstName": "Johan",
    "surname": null,
    "dateOfBirth": null,
    "gender": "MALE"
  },
  "organisationId": 1
}

Response (UNIQUE_MATCH):
{
  "status": "UNIQUE_MATCH",
  "confidenceScore": 65,
  "suggestion": {
    "matchToken": "abc123...",       // Opaque, time-limited
    "maskedName": "J*h** S***h",
    "gender": "Male",
    "ageRange": "35-40",
    "matchedFields": ["ID_NUMBER_EXACT", "GENDER"]
  },
  "suggestedAccessLevel": "READ"
}

Response (AMBIGUOUS):
{
  "status": "AMBIGUOUS",
  "candidateCount": 3,
  "message": "Multiple potential matches. Please provide more details.",
  "suggestedFields": ["surname", "dateOfBirth"]
}

Link by Token

POST /api/linked-people/link-by-token

Request:
{
  "matchToken": "abc123...",
  "linkType": "FAMILY",
  "organisationId": 1
}

Response:
{
  "linkedPersonId": 456,           // LinkedPerson.id (safe to expose)
  "accessLevel": "READ",           // Based on score
  "validTo": "2026-01-27T00:00:00Z"
}

Frontend UX Behaviour

This section specifies the user experience rules governing how and when the match popup is displayed, how it interacts with keyboard input, and how persistent indicators work.

Typing Pause Delay

The match dialog must not appear while the user is actively typing. A 2-second typing inactivity timer gates popup display:

  • API calls continue to fire per the existing debounce chain (500ms pre-score, 1.5s subsequent, 1s throttle)

  • When a UNIQUE_MATCH response is received and the reveal delay completes, the popup is held until 2 seconds have elapsed since the last keystroke

  • If the user resumes typing before the 2-second window, the popup is deferred, not discarded — it must appear once 2 seconds of inactivity have elapsed, however many times typing resumes in between

  • This timer is independent of the API debounce - it only controls popup visibility, not API call timing

The deferral is the subtle part. A revealed match is a pending obligation to show the popup, and typing only postpones it. Cancelling the pending display without re-arming it discards that obligation: the reveal has already fired and will not fire again for the same response, so the popup is lost for the remainder of the search — the user sees only the persistent indicator panel, whose "Show" button is not an obvious way forward. Any implementation that cancels a scheduled display on keystroke MUST reschedule it, not drop it.

Form input ──► PreScore (500ms) ──► API call (1.5s debounce) ──► Response
                                                                    │
                                                          Reveal delay (0-3s)
                                                                    │
                                                         Typing pause (2s) ──► Show popup

Dialog Keyboard Restrictions

When the match dialog is visible, keyboard interaction is restricted:

  • Escape - Closes the dialog (standard dismiss behaviour)

  • Enter - Activates the default action (e.g., Link button)

  • All other keys - Ignored; do not close or interact with the dialog

This prevents accidental dismissal when the user continues typing without noticing the popup. Implementation should intercept keydown events on the dialog and call event.preventDefault() / event.stopPropagation() for non-allowed keys.

Redundant Popup Suppression

The component tracks the highest match outcome that has been displayed to the user:

Tracked State Description

lastDisplayedAccessLevel

The access level of the last popup shown (READ or READ_WRITE)

lastDisplayedStatus

The response status of the last popup shown (UNIQUE_MATCH, AMBIGUOUS)

Suppression rules:

  • If the new response is UNIQUE_MATCH with the same or lower access level as previously displayed, do not re-show the popup

  • If the new response improves the outcome (e.g., READ → READ_WRITE), show the popup

  • If the response status changes (e.g., AMBIGUOUS → UNIQUE_MATCH), show the popup

  • The tracked state resets when the user explicitly clears the form, rejects or refines the match, or starts a new search

The reset on a new search is load-bearing, not housekeeping. Suppression is scoped to a single search run: it exists to stop one response re-presenting the same outcome the user has already seen, not to retire that outcome for the rest of the session. Dismissing the dialog — via its close control or Escape — leaves the tracked state populated, so if it is not cleared when the next search begins, every subsequent UNIQUE_MATCH at the same access level is judged redundant and the popup never returns. The user is left with the indicator panel as their only route forward.

Persistent Match Indicator Panel

When a match is found (any UNIQUE_MATCH response), a full-width indicator panel appears below the form card:

  • Displays the masked name from the suggestion (e.g., J*h S*h)

  • Shows a "Show" button that reopens the match dialog

  • Persists even after the dialog is dismissed

  • Removed when:

    • The user clicks "Not them" (reject match)

    • The form is cleared

    • A NO_MATCH response is received

  • The panel does not replace the dialog - it supplements it as a persistent reminder

The panel must not outlive the match it points at

The panel is a promise that "Show" will re-open a match. That promise has to hold, and the rules above alone do not make it hold: they say when the panel is removed, but say nothing about the suggestion behind it being discarded independently.

Issuing a new search must not destroy the current match before there is something to replace it with. If it does, the panel keeps rendering while the state that backs the dialog is gone, and "Show" opens an empty shell — during the whole in-flight window of every re-search, and permanently after a search that fails.

The invariant: a revealed match is retained until a later response replaces it.

  • A new search in flight does not clear it — the previous match stays showable until its replacement arrives

  • A UNIQUE_MATCH or AMBIGUOUS response replaces it

  • A NO_MATCH response clears it, along with the panel

  • A failed search leaves it intact. The last successful answer is still the best information available, and discarding it would cost the user a match they had already been shown because a later request happened to fail

This also removes the flicker that clearing-on-search would otherwise produce, which matters for a control whose stated purpose is to be a persistent reminder.

A retained suggestion carries a match token valid for TOKEN_EXPIRY_MINUTES. A user who re-opens a long-retained panel and confirms may therefore meet TOKEN_EXPIRED on redemption. That is a distinct concern from panel staleness and is handled by the link-by-token error path, not here.

Streamlined Add New Person Flow

When no match is found and the pre-score threshold is met:

  • An "Add New" button appears (replaces the current "Create New" button)

  • Clicking "Add New" directly persists the person using the data already in the search fields

  • No intermediate edit mode switch is required

  • The existing PeopleService.createPerson() API is called with the form data

  • On success, the user is navigated to the return URL

  • Form validation still applies - the button is disabled if required fields are invalid

This replaces the previous two-step flow: "Create New" (switch to edit mode) → "Save" (persist).

Match API Failure Handling

Every affordance that moves the user forward on this screen is downstream of a successful match response: "Add New" appears only on NO_MATCH, and the match dialog only on UNIQUE_MATCH / AMBIGUOUS. A failed match call therefore removes every exit from the screen at once, leaving only "Back" and "Clear". The screen must not depend on the match API succeeding in order to remain usable.

401 is never handled here. It belongs to AuthExpiredInterceptor, which re-establishes a guest session and returns the user to this route — see Which component owns which status. The component must not react to it.

All other statuses are the screen’s own to handle:

Status Retryable Behaviour

0, timeout, 5xx

Yes

Transient. Show an actionable message and a Retry control. This is the expected failure on race-day mobile connectivity and is the case the design optimises for.

400

Yes

Show the message and Retry. Retry is permitted because the user may correct the offending field between attempts.

403

No

Deterministic — permissions do not change between attempts. Go straight to the degraded state below without consuming retry attempts.

Retry re-fires the match for the current form values, bypassing the 1s throttle, and clears the failure state on success. It must also re-arm the search trigger: the "threshold first met" latch that fires the initial search is one-shot, so without an explicit reset a failed first search permanently disables automatic re-searching and only manual retry or a form edit can recover it.

Degraded state

After MATCH_CONSTANTS.MAX_MATCH_RETRIES consecutive failures (2), or immediately on 403, the screen additionally exposes Add New. The user is never permanently blocked from registering.

This is a deliberate trade against duplicate person records. Offering "Add New" when the match API is down invites the user to create someone who already exists — the outcome progressive matching exists to prevent. The ordering resolves it: a transient blip costs one click on Retry and creates nothing, and only a persistent failure surfaces the create path. Duplicates are recoverable through the merge tooling; a user who cannot register at all is not.

The counter resets on a successful match response and on Clear.

A failure invalidates the previous answer, but not the previous match

These pull in opposite directions and the distinction matters.

A prior NO_MATCH is an answer, and it was computed against the data the form held at the time. Once the user adds a field and the re-check fails, that answer no longer describes what is on screen — and the added field is precisely what might have produced a match. Leaving "Add New" on display off a stale NO_MATCH would let the user create a duplicate through the one door the degraded-state rule above exists to keep shut. A failure therefore clears the NO_MATCH state, and "Add New" is governed from then on by the retry counter alone.

A prior UNIQUE_MATCH is information, not an answer to the current question. Discarding it would take away a match the user had already been shown because a later request happened to fail, leaving them worse off than before they typed. A failure leaves it intact — see "The panel must not outlive the match it points at" below.

Records created through the degraded path are not currently flagged for duplicate review. Doing so would let operators triage exactly the population most at risk, but it needs a backend field and is out of scope here — track separately if the degraded path proves to be exercised in practice.

Rationale

Why Weighted Scoring?

Different fields have different uniqueness value:

  • ID Number is highly unique - strong proof of knowledge

  • Common names like "John" provide less assurance than "Bartholomew"

  • Email and phone are good identifiers but can be socially engineered

Weighted scoring reflects real-world identification confidence.

Why Uniqueness Requirement?

Returning multiple matches would:

  1. Reveal that multiple people with those characteristics exist (enumeration)

  2. Force UI to display a list (requiring personal data disclosure)

  3. Enable attackers to narrow down targets

By requiring exactly one match, we force the user to provide enough information to unambiguously identify the person - proving genuine knowledge.

Why Delayed Reveal?

The 3-second delay at threshold 50:

  1. Encourages users to add more fields (potentially reaching 80+ for better access)

  2. Prevents rapid probing (rate limiting enhancement)

  3. Gives legitimate users time to complete the form naturally

Why Opaque Tokens?

matchToken is:

  • Time-limited (expires after 5 minutes)

  • Single-use (invalidated after redemption)

  • Cryptographically secure (cannot be guessed)

  • Opaque (reveals nothing about the underlying data)

This ensures the client never learns the User.id, preventing:

  • IDOR (Insecure Direct Object Reference) attacks

  • Targeted enumeration based on sequential IDs

  • Correlation between sessions

Why Cross-Account Trust?

Real users may:

  • Create accounts via different auth methods over time

  • Forget they already linked family members in another login

Cross-account trust:

  • Reduces friction for legitimate users

  • Only applies to verified relationships

  • Includes time constraint (30 days) to prevent abuse

Prerequisites

Deduplication must complete before Progressive Matching goes live.

Current database duplicates would cause all matches to return "AMBIGUOUS", frustrating legitimate users. A separate Deduplication Epic should:

  1. Identify duplicate candidates (same ID number, similar names + DOB)

  2. Build admin merge UI for review

  3. Implement merge logic preserving relationships and audit trail

  4. Complete before Progressive Matching deployment

Security Controls Summary

Control Implementation

Enumeration Prevention

Uniqueness requirement - only returns if exactly one match

Proof of Knowledge

20-point frontend gate + 50-point backend threshold

Rate Limiting

Per-session limits on progressive match calls

Data Minimisation

Masked suggestions only; full data after verified link

PK Protection

User.id never exposed; matchToken and LinkedPerson.id only

Timing Attack Prevention

Consistent response times regardless of match status

Audit Trail

All progressive match attempts and token redemptions logged

Related Documentation

Implementation Checklist

  • PersonMatchScorer - Weighted scoring with configurable weights

  • PersonMatchConfig - YAML configuration for weights and thresholds

  • ProgressiveMatchResponse - Response DTOs for match states

  • PersonResource.progressiveMatch() - New endpoint

  • LinkedPersonResourceEx.linkByToken() - Token redemption endpoint

  • Frontend pre-scoring component

  • Delayed reveal UI with timer

  • Rate limiting aspect adaptation

  • Cross-account trust resolution

  • Progressive access level upgrade

  • Match API failure handling — retry, degraded state, search-trigger re-arm

Change History

Date Change

2026-08-01

Added "Match API Failure Handling" — the screen previously had no specified behaviour for a failed match call, and every forward affordance is downstream of a successful one. Sharpened "Typing Pause Delay" (a deferred popup must be rescheduled, not discarded) and "Redundant Popup Suppression" (the reset on a new search is load-bearing) after both were found to be implemented incorrectly during the 2026-07-31 stage investigation.

2026-08-01

Added "The panel must not outlive the match it points at" and "A failure invalidates the previous answer, but not the previous match", after review found the indicator panel could render while the state behind it had been discarded, so "Show" opened an empty dialog. The panel rules said when the panel is removed but nothing about the suggestion behind it, which left the two free to diverge.