Job

Source: src/AWS/DataBrew/Job.ts

An AWS Glue DataBrew job definition — either a PROFILE job that analyzes a dataset and writes a data-quality profile to S3, or a RECIPE job that applies a published recipe’s transformations and writes the result to S3. The definition is free and instant; job runs are billed per node-hour and are started with StartJobRun.

Profile Jobs

import * as AWS from "alchemy/AWS";

const profile = yield* AWS.DataBrew.Job("Profile", {
  type: "PROFILE",
  datasetName: dataset.datasetName,
  role: role.roleArn,
  outputLocation: { bucket: bucket.bucketName, key: "profiles/" },
  jobSample: { mode: "CUSTOM_ROWS", size: 1000 },
});

Recipe Jobs

const transform = yield* AWS.DataBrew.Job("Transform", {
  type: "RECIPE",
  datasetName: dataset.datasetName,
  recipeReference: { name: recipe.recipeName },
  role: role.roleArn,
  outputs: [
    {
      location: { bucket: bucket.bucketName, key: "curated/" },
      format: "CSV",
      overwrite: true,
    },
  ],
});