MatchingWorkflow

Source: src/AWS/EntityResolution/MatchingWorkflow.ts

An AWS Entity Resolution matching workflow — a data-processing job definition that matches and deduplicates records across Glue table input sources using rule-based or ML-powered matching, writing matched record groups to S3.

The workflow definition itself is cheap and instant; a matching RUN (StartMatchingJob) processes the full input and takes many minutes.

Creating Workflows

Rule-based matching over a Glue table

import * as AWS from "alchemy/AWS";

const workflow = yield* AWS.EntityResolution.MatchingWorkflow("Dedupe", {
  inputSourceConfig: [
    { inputSourceARN: table.tableArn, schemaName: schema.schemaName },
  ],
  outputSourceConfig: [
    {
      outputS3Path: `s3://${bucket.bucketName}/matches/`,
      output: [{ name: "id" }, { name: "email" }],
    },
  ],
  resolutionTechniques: {
    resolutionType: "RULE_MATCHING",
    ruleBasedProperties: {
      rules: [{ ruleName: "ByEmail", matchingKeys: ["email"] }],
      attributeMatchingModel: "ONE_TO_ONE",
    },
  },
  roleArn: role.roleArn,
});

ML-powered matching

const workflow = yield* AWS.EntityResolution.MatchingWorkflow("MlDedupe", {
  inputSourceConfig: [
    { inputSourceARN: table.tableArn, schemaName: schema.schemaName },
  ],
  outputSourceConfig: [
    {
      outputS3Path: `s3://${bucket.bucketName}/ml-matches/`,
      output: [{ name: "id" }],
    },
  ],
  resolutionTechniques: { resolutionType: "ML_MATCHING" },
  roleArn: role.roleArn,
});