Book Importer: the build
Engineering plan, with every load-bearing claim checked in source
Engineering plan, with every load-bearing claim checked in source
Build a converter that takes a brokerage's existing book from whatever source holds it and lands it in Plansight as real groups and plans: who the carrier is, what the rates are, and what the plan attributes are. Import only. There is no write-back and no live sync, and that single constraint is what keeps this a quarter of work instead of a multi-year integration.
The most important thing to understand before you start
Three of the five stages already exist in production. Open Plan reads the documents. V1's
PlanfactService converts Open Plan output into V1 plan data. V1's benefit history store
accepts and validates those plans. We are writing the converter and the review layer, and
wiring them to things that already run. The failure mode on this project is somebody
deciding to port or re-implement Open Plan or PlanfactService. Do not.
Scope, decided by Steve. Three things, and one deliberate exclusion.
The conversion answers exactly three questions per plan: who is the carrier, what are the rates, and what are the plan attributes. That is the whole target.
Contributions are explicitly out of scope. They are not plan data in Plansight, they live in the presentation layer, and they are set per scenario rather than being a fact about the plan. Pulling them into the converter would mean inventing a target that does not exist. If a source carries contribution data, ignore it. Do not add it back without a decision.
What each one does, and what already exists.
SharePoint via Graph, BrokerageBuilder via REST, or a spreadsheet.
To build/docid then /extract-all, async callbacks, cost telemetry.
Claude normalizes into Employer Book JSON with provenance.
To buildOpen Plan JSON to V1 fields, then the benefit history store.
ExistsLow confidence rows surfaced, golden set hand-verified.
To buildEach adapter's only job is to produce a pile of documents plus whatever structured hints the source offers. They are deliberately thin.
| Adapter | Transport | Access | Status |
|---|---|---|---|
| Document library SharePoint |
Microsoft Graph, read only, Sites.Selected |
Routine IT ask on the customer side | No blocker. Build first. |
| BrokerageBuilder Zywave AMS |
An extraction agent driving credentials, not a typed client. REST surface documented; OAuth 2.0 with scope api.bkb:read where a token is granted. |
Baldwin confirmed they run it and is pursuing tokens for us. ThreeFlow had direct back-end extract access, so the precedent exists. | Unblocked by the customer. See the BKB Extraction plan. |
| Spreadsheets employer and plan lists |
File upload | None | No blocker |
Scope the SharePoint pull deliberately: current plan year only, dedupe by content hash, and
skip census and enrollment files, which carry PII we have no reason to ingest for a
plan conversion. Folder names are a useful employer-matching signal, cross-checked against what
/docid reports.
Open Plan already handles 11 product types plus statutory NY and NJ forms, and it is runnable locally for the spike with poetry and a dev key.
/docid classifies a PDF: which products it contains, carrier, employer, and page ranges./extract-all pulls the facts, with cross-product caching so a multi-line document is not read repeatedly.include_sources returns bounding boxes, which is what makes every converted value traceable back to the page it came from./docid exists in Open Plan but V1 has never called it
Do not assume this step is wired up. PlanfactService::apiCall() takes
$planType as its first parameter, meaning V1 tells Open Plan what the
document is rather than asking. The $documentId argument in that same signature is V1's
own correlation id for the async callback, not Open Plan classification. Grep confirms no
/docid call anywhere in the service.
Classification is therefore net-new integration work for the importer, and it is the single highest-value new capability in the whole pipeline. Extraction and verification are not the gap. The plan builder already uploads a document, extracts the plan, and puts it in front of a user to verify. What it requires is that a person tag the document to an employer and a line of coverage first, which is fine per plan and impossible across a book. Stage 2 is marked as existing because the engine exists, not because this path is built.
Decided 2026-08-09, do not re-litigate
The Open Plan schema is the canonical extracted-facts layer for every source, not just documents. The BrokerageBuilder adapter normalizes that tenant's free-text benefit items into Open Plan schema too, so the converter has exactly one target vocabulary. The reasoning is that Open Plan is the engine the company teaches to read benefit documents, and it eventually moves into Phoenix. V1 is the near-term landing target only, so V1 field derivation happens at landing rather than being baked into the stored Book. When Open Plan moves into Phoenix, converted books re-land there from the same files with no re-extraction. The V1 emitter is the only throwaway piece.
A standalone service, following the rating-service pattern rather than living inside V1. It takes Open Plan facts and emits the neutral Employer Book: one file per employer, carrying Open Plan schema facts, per-value provenance and confidence, and the raw source values that produced them.
Claude Opus 5 with structured outputs, so the schema is enforced at the tool-call layer and there are no parse failures to handle.
Conversions are overnight jobs by nature, which puts the whole workload on the 50 percent batch discount.
The field spec repeats on every group in the book. Caching that prefix is roughly a 90 percent saving on the largest part of each request.
Every value carries where it came from and how confident the model was. That is what feeds the review queue and the customer-facing source annotation.
First task, before any conversion code
Build the field-spec exporter. A script run against the local V1 repo that dumps
Plantypes::tableColumns() output per line of coverage to JSON. It is both the converter's
vocabulary source of truth and its cached prompt prefix. An external service cannot call the PHP
enumerator at runtime, so this has to be a build artifact. It is small and it unblocks everything else.
app/Services/PlanfactService.php is 7,457 lines and already does this job in production. It
carries an apiCall() client plus a parseQuote dispatcher and 11 product parsers
(medical, dental, vision, group life, voluntary life, short term disability, long term disability,
accident, critical illness, hospital indemnity, stop loss), handling network mapping, critical illness
tobacco rate bands, and bounding box attachment. It is the live AI-quote path behind
AIQuoteJob and QuoteController.
Reuse it through V1. Never port it. The converter should emit extraction-format payloads so PlanfactService normalizes server-side, which also means the import benefits from every future fix to that parser.
Corrected 2026-08-09: the bulk-Excel bridge does not exist
An earlier version of this plan assumed V1 could take a spreadsheet of benefit history and import it.
It cannot. routes/web.php:56 is quoteBenefitHistoryQuotesUpload, a
captcha-gated document attachment upload for external carrier links. It never parses a
spreadsheet into data, and no other code path in V1 does either.
The real landing path is a scripted operator session driving
POST rest/app/group/{groupId}/benefitHistory/{quoteId?}
(routes/web.php:922, BenefitHistoryController@store). This needs zero V1
code changes to demo, and every plan still passes Quote::validateQuoteData() server-side,
which the imagined Excel bridge would have bypassed. Once the shape is proven, one dev card adds a
real import endpoint.
The single most important shape decision in the Employer Book schema.
The naive read of this project is that it lands a pile of plan records. That would be a mistake, because V1 already thinks in packages and the importer should emit them. Two existing mechanisms prove the pattern, and both are live on master today:
RfpTemplateController copies a whole RFP, its plan groups, and its presentation data,
then sets groupId = 'none' (:228), detaching the bundle from any
employer. It is stamped with brokerageId, the list query carries no user
filter, and effective-date attributes are skipped on copy, so it is brokerage-wide and it does
not expire.
QuoteDefault extends Quote, so a library entry is a whole plan with benefits and
rates rather than a stub. It carries a user-given name, is Elasticsearch-indexed, and
isShared plus shareWithBrokerage make it visible across the brokerage.
Design consequence for the Book schema
An Employer Book is the same shape of object as an RFP template: a coherent set of plans that can be detached from the employer it came from and applied elsewhere. Schema and emitter must both respect that.
Concretely: an employer's plans land as one bundle with a group boundary, not as
independent rows that happen to share a groupId. Every plan must additionally carry
enough on its own (name, carrier, type, benefits, rates) to stand up as a library entry without the
bundle around it. Get this right at the schema stage and the export path, the template path, and the
library path all fall out for free. Get it wrong and every one of them becomes a retrofit.
This also closes the loop with work already in the product. We import and export RFPs. If the Book is bundle-shaped, then a converted employer can seed an RFP directly, go to market, take quotes back in, and produce the presentation, with no intermediate re-entry step. The conversion stops being a migration and becomes the first step of the normal workflow.
Verified against master at a8e7f76c89. Re-check line numbers before writing code.
| Constraint | What it means for the importer |
|---|---|
| No attribute or rate tables | Plan design is schemaless string bags in DynamoDB, and rates are roughly 60 flat columns on the 17,675-line app/Models/Quote.php. There is no relational structure to target. |
| Attribute names derive at runtime | The only correct enumerator is Plantypes::tableColumns() at app/Helpers/Plantypes.php:1483. Secondary lists elsewhere in the codebase have drifted. Never use the map at Quote.php:657. |
| 36 plan types | The registry is config/plantypes.php. Anything outside it lands as a plan shell plus documents rather than failing the import. |
| Validation is server-side | Every import must pass Quote::validateQuoteData(). Treat a rejection as a review-queue row, not an exception. |
| Stop loss rates are mutated in place | V1's cost calculation folds fees into the stored columns (Quote.php:9489-9495). Write raw carrier rates only, or the numbers drift every time cost is recalculated. |
| Rate tier fields are paired | rateTierSchedule (string) and rateTiers (int) must both be set. Setting one is a silent data problem. |
| No smoker rate dimension | V1 has none. Source data that carries smoker-differentiated rates needs an explicit collapse rule decided once and documented. |
| Write surfaces are session-authed | The public API v1 is read only. Writes go through session-authed web routes, which is why the first landing path is a scripted operator session. |
Demoable in weeks. Production-grade in about a quarter.
Field-spec exporter, then employer groups and plan shells: names, types, carriers, plan year, policy numbers, subscriber counts, statuses. Includes the two per-source mapping tables with review screens, plan types and carriers, the latter expecting fuzzy name matching with an A.M. Best cross-check. Deliverable is already a sales demo: here is your whole book in Plansight.
Rates, rate guarantees, fees. Fixed, tiered, and volume rate shapes are clean; age-banded complex rates need parsing; self-funded follows the raw-rates-only rule above. Per-source tier-name mapping reuses the Phase 1 review screen pattern. Borrow the transferable BenefitPoint patterns here: capability probe, graceful degradation, audit log.
Item-name matching of source benefit items to V1 field keys, value normalization into V1's typed token sets, and cross-validation against document extraction wherever both a document and a structured source exist for the same plan. Review queue UI for low-confidence rows. Golden-set verification: hand-check a fixed number of plans per line of coverage before declaring any tenant converted.
Write-back to the source AMS. Keeping the old system alive post-conversion recreates BenefitPoint-scale cost for a customer we are actively trying to move off that system.
The in-house yardstick is 324 files and roughly 55,200 lines.
| Axis | BenefitPoint | Book Importer |
|---|---|---|
| Direction | Bidirectional push and pull with renewal matching | Import once, re-runnable, no push |
| Transport | Legacy SOAP plus REST, per-customer WSDL quirks | One modern REST API or a document pull |
| Rate mapping | Exact per-ID fidelity in both directions. RateData.php alone is 6,992 lines. | Six shapes to V1 columns, one direction, fuzzy with review is acceptable |
| Attribute mapping | Per-attribute IDs wired into 35 config files | Name-level matching, AI normalization, document fallback |
| Failure tolerance | A failed push corrupts the system of record | A failed import is a row in a review queue |
curl and parse paths and components.schemas directly. The same
applies to any large JSON spec.basePlanTypeId and PlanType.amsCode. Only the six rate structures are fixed.
Benefit values are free text up to 1000 characters.~/Documents. Recursive case-insensitive greps over
those directories time out past 120 seconds. Scope them narrowly or background them.parseQuote* output lands cleanly in quoteBenefitHistory. It extends
Quote, so this is expected to work, but the spike must prove it rather than assume it.Plantypes::tableColumns()
for every one of the 36 types, including the ones the AI reader does not cover.Expect the "rebuild it natively in V1" conversation
Roster hit exactly this, and the pushback landed on rebuilding natively. The dev card for the import endpoint should cite that precedent directly and be explicit that the recommendation here is an external service first, promoted to a V1-native package only if conversion volume justifies it.
Verified in source on August 10, 2026 against ~/Code/plansight-v1 master
a8e7f76c89: PlanfactService at 7,457 lines with 11 product parsers plus a dispatcher;
Plantypes::tableColumns() at Plantypes.php:1483; 36 entries in
config/plantypes.php; benefitHistory.store at routes/web.php:922;
quoteBenefitHistoryQuotesUpload at routes/web.php:56;
Quote.php at 17,675 lines.
Full working documents live in the Plansight Brain vault under BrokerageBuilder Integration/.