Ruleset

Source: src/AWS/DataBrew/Ruleset.ts

An AWS Glue DataBrew ruleset — a set of data-quality rules bound to a dataset. Attach it to a profile job via validationConfigurations to produce pass/fail validation results alongside the data profile.

Creating Rulesets

Data-Quality Rules for a Dataset

import * as AWS from "alchemy/AWS";

const ruleset = yield* AWS.DataBrew.Ruleset("Quality", {
  targetArn: dataset.datasetArn,
  rules: [
    {
      name: "no-missing-ids",
      checkExpression: "AGG(MISSING_VALUES_PERCENTAGE) == :val1",
      substitutionMap: { ":val1": "0" },
      columnSelectors: [{ name: "id" }],
    },
  ],
});

Validate in a Profile Job

const profile = yield* AWS.DataBrew.Job("Profile", {
  type: "PROFILE",
  datasetName: dataset.datasetName,
  role: role.roleArn,
  outputLocation: { bucket: bucket.bucketName, key: "profiles/" },
  validationConfigurations: [{ rulesetArn: ruleset.rulesetArn }],
});