Proof of concept

koraiPivot

A control plane for data migrations: what has to move from one world to another, in which order, what made it across, what was refused and why. koraiPivot writes no new connectors — it builds on those of the koraiCore platform and adds what a migration project actually demands: replayable runs, a resume point, rejects you can fix and replay, and the project itself in the same place.

Built on koraiCoreSelf-hostableYour data stays with you

The problem

A data migration rarely fails on the transfer. It fails on everything around it.

  • The script that loaded the data the first time cannot say what it wrote, nor what it left behind.
  • Run again after a fix, it duplicates what had already made it across — or the target has to be emptied and the whole thing redone.
  • Interrupted on the four hundred thousandth row, it starts again at the first.
  • Refused rows end up in a log nobody reads: the gap surfaces months later, while looking for a customer record.
  • The project itself — waves, freeze windows, cutover, sign-off — lives in a spreadsheet alongside, with no link to what actually ran.

What koraiPivot does today

Everything below works on record migrations — databases, files, APIs, search engines: whatever the koraiCore platform connectors can read and write —, on documents from the document management system, on a REST target described by configuration, and on the transports another application makes available. The screens shown in a demonstration run on an entirely fictional dataset: an invented company, invented systems, synthetic runs and rejects.

A plan, and what it declares

A migration plan names a source, a target, a field mapping and a replay rule. It also declares the batch size, how many rows the transform handles in parallel, the share of rejects beyond which a run must stop, and whether it is a dry run. The plan is inert: only a run writes. It can be cloned — the second wave is usually the first with another filter — and it leaves the working catalogue by being archived rather than deleted, because its runs are the record of what happened to production data.

The plan catalogue, by family and by mode

Four ways to read the source

Full: the whole source, on every run. Partial: a declared subset — one wave, a list of identifiers, the corrected rejects. Incremental: whatever changed since the marker left by the last successful run, for the nightly syncs of the period when both systems are live. Continuous: a stream, where the source can produce one. All four compose on a real project, and replay in whatever order you need.

The run history, across every plan

Replay without duplicating

This is the question that settles everything else. Three answers, declared by the plan: the natural key of the target, which updates instead of inserting; the pivot log, which remembers which target object each source object became; or none at all, reserved for an empty target and a single run — and which has to be chosen explicitly, never by default.

The no-duplicate rule is declared on the plan

A dry run that writes nothing

A dry run reads and transforms for real, then writes nothing. It produces the same counters, the same rejects and the same diagnostics as a live run: a dress rehearsal, not a preview. A dry run can also be forced when launching a live plan; the reverse cannot happen: going back to real writes means editing the plan.

A dry run is a switch on the plan

A resume point at every batch

Every committed batch writes its resume point: how far the source has been read, the date reached and the batch number. An interrupted run — network failure, restart, error threshold crossed — resumes where it stopped instead of starting over. That point only advances once the batch is safely written: an interruption makes the run re-read rows, never skip them.

Batch size, hence the granularity of resumes

An error threshold that really stops

Beyond the share of rejects declared by the plan, the run stops on a clean batch rather than producing wrong data at scale. A plan can demand a stop on the first reject, or never stop at all — that is a choice, and it is written into the plan.

The error threshold, as a share of rows read

Rejects you can act on

Every refused row keeps the phase where it died — read, transform, validate, resolve, write —, a stable reason code to group by, a readable message and a diagnostic. The dashboard breaks down by reason the rejects of the last run that ended badly: four hundred rejects sharing one cause are one fix to make, not four hundred decisions to take. The original row is kept exactly as it was read, so the replay has something to start from. Viewing or correcting it takes a dedicated permission: every read and every correction is written to the audit trail, fields the plan declares sensitive stay masked without a second right, and a correction requires a comment. The corrected version is the one the replay picks up; the original is kept.

Rejects grouped by reason, phase by phase
A run's rejects, row by row

Checks declared by the plan

Between transformation and writing, the plan chains the checks it declares, without knowing anything about what the records represent. A confidence rule rejects an extracted field below the set threshold; the koraiCore platform business rules apply record by record; dedup keys skip content already delivered, including by the original plan when this one is a corrective clone of it; a reference resolution finds in the target the entity a field points to, creates it if the plan allows, or rejects. Every successful write records the link between source object and target object in the same transaction as the resume point.

Documents as a source, and other targets

A migration does not always start from a database. A plan can read documents from the platform document management system, optionally run through field extraction against a schema it supplies, with the confidence of each field. Files dropped into a watched folder arrive there by themselves, and a continuous plan wakes up when they do. On the target side, a REST API is described by configuration; another application can also offer its own sources and targets, which koraiPivot reaches through the platform relay. The dashboard shows what the instance can actually read and write.

What the instance can read and write

The project in the same place

Projects, waves, tasks and defects are koraiCore objects with their own lifecycles: a project runs from scoping to cutover and then verification, a wave from planning to the final check on discrepancies — or to rollback. Lists, kanban, timeline and "my work" build on them, with assignments, a timer and history. A migration plan attaches to its project: what ran and what remains to be decided no longer live in two different tools.

Anomalies, with their life cycle
The timeline of projects and waves

Field mappings, and the return migration

The transform is not reinvented: field mappings are the platform ones, shared with the other applications, with their validation, their lookup tables and — what matters on the day of a rollback — their inversion. A return plan is the outbound plan with its mapping inverted.

Mappings, with their declared source and target

The pivot log, object by object

A run’s counters say four hundred thousand rows made it across. The pivot log says that this ticket became that one, through which mapping and in which direction. It is the trace that answers a precise question months later, and it is also what the pivot-log replay rule reads.

Every object migrated, on both sides

One API, and access for assistants

Everything the screens do goes through the same GraphQL API, under the same permissions and the same separation between organisations. Three tools exposed to assistants give the state of plans, runs and rejects, read-only: an agent can report where the migration stands without being able to launch anything.

What the exploration showed

koraiPivot came out of an internal observation: every data migration rewrote its own script, and none of them left a usable trace. The question was whether the koraiCore platform made it possible to turn that into a tool.

The plumbing was already there

Source connectors, field mappings, tenant separation, permissions, scheduling, audit trail: koraiPivot inherits them from the koraiCore platform instead of rewriting them. The effort went into what was genuinely missing: resuming after an interruption, replaying without creating duplicates, making rejects usable, and running the project.

See the koraiCore platform

What the domain imposed

A migration is judged by what it does when it fails. A batch that only advances its resume point once committed, a dry run that is genuinely dry, a threshold that stops the run rather than letting it produce wrong data: these are not ergonomic options, they are the only rules that make a migration replayable.

Nothing goes to a third party

The application is hosted wherever you want, credentials for the source and target systems stay in your configuration, and migrated data passes through no outside service. A data migration is precisely the moment when an entire information estate travels through a single route: that route is yours.

Frequently asked questions

What is koraiPivot?

koraiPivot is data migration software written by Koraisoft on its koraiCore platform. It models a migration project — plans, batched runs, resume points, rejects, a final check on discrepancies — and carries it from the first dry run to cutover. It is a proof of concept: software explored internally, not a commercial offering.

Can koraiPivot be used or bought?

No. koraiPivot is a Koraisoft proof of concept: it is not distributed, not supported, and no roadmap commits to it. There is no licence to buy, no trial, no hosted service. If the subject interests you, getting in touch is the only way in today.

Which migrations can koraiPivot handle today?

koraiPivot handles record migrations: whatever the koraiCore platform connectors can read and write — relational databases, document stores, files, APIs, search engines. It also reads documents from the document management system, with field extraction, writes to a REST API described by configuration, and uses the sources and targets another application makes available. Issue-tracker migrations build on the same connectors and on the pivot log.

How does a migration differ from a continuous data flow?

A continuous flow links systems that both stay alive; koraiPivot runs a project that has an end, where one of the two worlds has to be switched off. Hence what it carries on top: running the project, the waves, partial replay of corrected rows, the final check on discrepancies and the cutover decision.

What happens to a row koraiPivot refuses?

It is kept as a reject, with the phase where it failed (read, transform, validate, resolve or write), a stable reason code, a message and a diagnostic, along with the source record as it was read. Rejects group by reason and can be marked as dealt with. With the dedicated permission, the data of a reject can be viewed and corrected — every access is audited, sensitive fields stay masked — and a partial run replays the corrected rows without touching the source or re-reading the rest.

Discuss it with us

Data migration, moving off a legacy system, or more generally what can be built on the koraiCore platform: if any of these topics interests you, we would be glad to discuss it.

Get in touch

The screenshots on this page and the online demo run on fictional, deliberately small datasets: they illustrate only part of what the product does, and some features are limited or switched off in the demo environment.