DataHubApply

This source is a helper over CLI so people can use the helper to apply various metadata changes to DataHub via Managed Ingestion

CLI based Ingestion

Config Details

Options

Note that a . is used to denote nested fields in the YAML recipe.

Field	Description
domain_apply array	List to apply domains to assets
domain_apply.DomainApplyConfig DomainApplyConfig
domain_apply.DomainApplyConfig.domain_urn string	Default:
domain_apply.DomainApplyConfig.assets array	List of assets to apply domain hierarchichaly. Currently only containers and datasets are supported
domain_apply.DomainApplyConfig.assets.string string
owner_apply array	List to apply owners to assets
owner_apply.OwnerApplyConfig OwnerApplyConfig
owner_apply.OwnerApplyConfig.owner_urn string	Default:
owner_apply.OwnerApplyConfig.assets array	List of assets to apply owner hierarchichaly. Currently only containers and datasets are supported
owner_apply.OwnerApplyConfig.assets.string string
tag_apply array	List to apply tags to assets
tag_apply.TagApplyConfig TagApplyConfig
tag_apply.TagApplyConfig.tag_urn string	Default:
tag_apply.TagApplyConfig.assets array	List of assets to apply tag hierarchichaly. Currently only containers and datasets are supported
tag_apply.TagApplyConfig.assets.string string
term_apply array	List to apply terms to assets
term_apply.TermApplyConfig TermApplyConfig
term_apply.TermApplyConfig.term_urn string	Default:
term_apply.TermApplyConfig.assets array	List of assets to apply term hierarchichaly. Currently only containers and datasets are supported
term_apply.TermApplyConfig.assets.string string

Schema

The JSONSchema for this configuration is inlined below.

{
  "title": "DataHubApplyConfig",
  "type": "object",
  "properties": {
    "domain_apply": {
      "title": "Domain Apply",
      "description": "List to apply domains to assets",
      "type": "array",
      "items": {
        "$ref": "#/definitions/DomainApplyConfig"
      }
    },
    "tag_apply": {
      "title": "Tag Apply",
      "description": "List to apply tags to assets",
      "type": "array",
      "items": {
        "$ref": "#/definitions/TagApplyConfig"
      }
    },
    "term_apply": {
      "title": "Term Apply",
      "description": "List to apply terms to assets",
      "type": "array",
      "items": {
        "$ref": "#/definitions/TermApplyConfig"
      }
    },
    "owner_apply": {
      "title": "Owner Apply",
      "description": "List to apply owners to assets",
      "type": "array",
      "items": {
        "$ref": "#/definitions/OwnerApplyConfig"
      }
    }
  },
  "additionalProperties": false,
  "definitions": {
    "DomainApplyConfig": {
      "title": "DomainApplyConfig",
      "type": "object",
      "properties": {
        "assets": {
          "title": "Assets",
          "description": "List of assets to apply domain hierarchichaly. Currently only containers and datasets are supported",
          "type": "array",
          "items": {
            "type": "string"
          }
        },
        "domain_urn": {
          "title": "Domain Urn",
          "default": "",
          "type": "string"
        }
      },
      "additionalProperties": false
    },
    "TagApplyConfig": {
      "title": "TagApplyConfig",
      "type": "object",
      "properties": {
        "assets": {
          "title": "Assets",
          "description": "List of assets to apply tag hierarchichaly. Currently only containers and datasets are supported",
          "type": "array",
          "items": {
            "type": "string"
          }
        },
        "tag_urn": {
          "title": "Tag Urn",
          "default": "",
          "type": "string"
        }
      },
      "additionalProperties": false
    },
    "TermApplyConfig": {
      "title": "TermApplyConfig",
      "type": "object",
      "properties": {
        "assets": {
          "title": "Assets",
          "description": "List of assets to apply term hierarchichaly. Currently only containers and datasets are supported",
          "type": "array",
          "items": {
            "type": "string"
          }
        },
        "term_urn": {
          "title": "Term Urn",
          "default": "",
          "type": "string"
        }
      },
      "additionalProperties": false
    },
    "OwnerApplyConfig": {
      "title": "OwnerApplyConfig",
      "type": "object",
      "properties": {
        "assets": {
          "title": "Assets",
          "description": "List of assets to apply owner hierarchichaly. Currently only containers and datasets are supported",
          "type": "array",
          "items": {
            "type": "string"
          }
        },
        "owner_urn": {
          "title": "Owner Urn",
          "default": "",
          "type": "string"
        }
      },
      "additionalProperties": false
    }
  }
}

Code Coordinates

• Class Name: datahub.ingestion.source.apply.datahub_apply.DataHubApplySource
• Browse on GitHub

Questions

If you've got any questions on configuring ingestion for DataHubApply, feel free to ping us on our Slack.