How to build an agentic security master (secmaster)

Quick answer

An agentic security master is a set of warehouse tables kept current by scheduled agents. Each morning a Nori agent loads the vendor reference files you already license and your custodian files, validates identifiers, cross-matches CUSIP, ISIN, SEDOL, FIGI and ticker, and proposes one golden record per security. Every field records which source it came from. Conflicts it cannot resolve by your written rules go to an exceptions queue for a person in operations to decide.

What the agent does and what it leaves to people

A security master fails quietly. A CUSIP changes after a corporate action, one custodian picks it up on Monday and a vendor file on Wednesday, and for two days positions do not reconcile. The matching work is rule-heavy, repetitive and well documented, which makes it a good fit for an agent. The judgment calls are not, so the design keeps them with people.

• The agent does: file loading, identifier validation, cross-matching, applying your survivorship rules, writing lineage, drafting exceptions with evidence, and re-running on a schedule.
• People do: write the survivorship rules, decide exceptions, approve changes to the rules, and approve anything that changes a record a downstream system already uses.
• Reference data comes from your sources. The agent works on feeds you already license and on files your custodians send you. Nori does not supply or redistribute vendor reference data, and identifier licensing terms remain between you and each provider.

Table schemas

Five tables are enough to start. Raw files stay untouched in a landing schema; everything below is derived and reproducible.

-- one row per security, the golden record
create table secmaster.security (
    security_id        varchar primary key,   -- internal, never reused
    security_name      varchar not null,
    asset_class        varchar not null,      -- equity, govt_bond, corp_bond, fund, ...
    issuer_id          varchar,               -- fk to secmaster.issuer
    currency           char(3),
    country_of_risk    char(2),
    primary_exchange   varchar,               -- ISO 10383 MIC
    maturity_date      date,
    coupon_rate        numeric(9,6),
    status             varchar not null,      -- active, matured, delisted, pending_review
    valid_from         date not null,
    valid_to           date                   -- null = current version
);

-- every identifier ever seen for a security, with validity dates
create table secmaster.security_xref (
    security_id   varchar not null,
    id_type       varchar not null,           -- CUSIP, ISIN, SEDOL, FIGI, TICKER
    id_value      varchar not null,
    exchange_mic  varchar,                    -- required for TICKER and SEDOL
    valid_from    date not null,
    valid_to      date,
    source        varchar not null
);

-- which source supplied each field of the golden record
create table secmaster.field_lineage (
    security_id   varchar not null,
    field_name    varchar not null,
    field_value   varchar,
    source        varchar not null,           -- vendor_a, custodian_1, manual
    source_file   varchar not null,
    source_row    integer,
    rule_applied  varchar not null,           -- survivorship rule id
    decided_by    varchar not null,           -- 'agent' or a person
    decided_at    timestamp not null
);

-- conflicts the rules could not resolve
create table secmaster.exception (
    exception_id  varchar primary key,
    security_id   varchar,
    exception_type varchar not null,          -- id_conflict, field_conflict, no_match, ...
    detail        variant not null,
    status        varchar not null,           -- open, resolved, deferred
    assigned_to   varchar,
    opened_at     timestamp not null,
    resolved_at   timestamp,
    resolution    varchar
);

Two design points matter for matching. Identifiers are stored with validity dates, because a CUSIP or ticker can be reassigned and a historical position must still resolve. And a ticker is never a key on its own: it is unique only with an exchange and a date.

How to build it, step by step

1. Land the files. Point your existing SFTP or bucket delivery at a landing area the agent's role can read: vendor reference files and each custodian's security and position files. Keep every file; lineage refers to them by name.
2. Write the survivorship rules. A short YAML file states which source wins for each field, for example the vendor feed for maturity_date and coupon_rate, with custodian files used only to break ties. Store it in the repository with operations leads as required reviewers.
3. Ask the agent to build the loaders and the matcher. @nori build loaders for the vendor_a and custodian_1 file layouts in docs/layouts, then the matching job described in secmaster/README.md. Open a PR with tests.
4. Validate identifiers before matching. The job checks length, character set and check digit for CUSIP, ISIN and SEDOL. A value that fails is not used as a match key and is logged against its source file.
5. Match in a fixed order. FIGI and ISIN first, then CUSIP, then SEDOL with exchange, then ticker with exchange and date as supporting evidence only. A US ISIN is checked against the CUSIP embedded in it.
6. Apply survivorship and write lineage. For every field the job writes the chosen value, the source, the file, the row and the rule id to secmaster.field_lineage.
7. Queue what does not resolve. Anything outside the rules becomes a row in secmaster.exception and a line in the morning Slack summary.
8. Schedule it. A Nori automation runs the job every business morning after files arrive and again when a custodian file lands late.

The survivorship file is short on purpose:

# secmaster/survivorship.yml
source_priority:
  default:         [vendor_a, vendor_b, custodian_1, custodian_2]
  security_name:   [vendor_a, custodian_1]
  maturity_date:   [vendor_a, vendor_b]
  coupon_rate:     [vendor_a, vendor_b]
  country_of_risk: [vendor_b, vendor_a]

auto_accept:
  - new security where at least two sources agree on ISIN and currency
  - name changes that differ only in case, punctuation or legal suffix

always_queue:
  - any identifier change on a security with open positions
  - conflicting maturity_date or coupon_rate
  - a security that matches two existing golden records
  - asset_class changes

Worked example: one exception from the morning run

The automation runs on 0 6 * * 1-5 in the operations time zone. Most mornings the summary is three lines. This one has an exception. All identifiers below are made up for illustration.

Nori  06:14
Secmaster run for 2026-11-11: 18,402 securities checked.
37 new records auto-accepted, 112 fields updated by rule, 1 exception.

EXC-2026-1111-001  id_conflict  Northfield Industrial Corp, common stock
  custodian_1 positions file (cust1_pos_20261110.csv, row 2291)
      CUSIP 66611T208   ticker NFIC   qty 4,150
  vendor_a reference file (va_ref_20261110.dat)
      CUSIP 66611T109   ticker NFIC   ISIN US66611T1097
  golden record SEC-0048213 holds CUSIP 66611T109 since 2019-03-04.

Likely cause: vendor_a corporate actions file from 2026-11-07 lists a
1-for-5 reverse split for this issuer effective 2026-11-10 with a new
CUSIP. The reference file has not picked it up yet. Custodian quantity
fell from 20,750 to 4,150, which matches the ratio.

Proposed: end-date CUSIP 66611T109 at 2026-11-09, add 66611T208 from
2026-11-10, same security_id. Rule: always_queue (identifier change
with open positions). Reply approve, reject, or defer.

An operations analyst replies approve in the thread. The agent writes the cross-reference rows, records the analyst as decided_by in lineage, closes the exception, and reruns the position reconciliation for that custodian. If the analyst rejects, nothing changes and the exception stays open with the reason.

The lineage for that one security then reads:

Keeping the master current

Three automations keep it from going stale:

• Daily run. Loads, matches, applies rules, posts the summary. Silent sections are omitted.
• Late-file webhook. Your file transfer job calls a Nori webhook when a custodian file arrives after the cut-off. The agent reruns only that source.
• Weekly hygiene. Lists securities with no position and no reference update in a year, open exceptions older than five days, and rules that fired most often. Frequent always_queue hits on one pattern are a prompt to write a new rule, through a reviewed pull request.

Resolved exceptions are saved to shared memory with their cause. The next time a reverse split produces the same pattern, the agent cites the earlier case in its proposal. It still queues it, because the rule says so.

A reliable master is what makes downstream work possible. The fund look-through table maps every holding to security_id, and the same pattern supports custodian reconciliation and NAV data checks described in AI agents for asset managers. OpenFIGI publishes FIGI mappings openly, which makes FIGI a useful neutral join key next to the identifiers you license.

Frequently asked questions