This command configures parsing for Kafka events. Fields in the the event message are specified for parsing, as well as a database table in which the parsed message information is saved. The table must already exist. Columns are created in the target table for the fields that are parsed, if they do not exist already.
Usage
vcli: admin> blobexpansion create --database-name DATABASE
--table-name TABLE
--target-table-name TABLE
--arrow-schema ARROW_SCHEMA
[--tenant-id TENANT]
[--target-table-schema TARGET_SCHEMA]
[--expansion-format EXPANSION_FORMAT]
[--source-column-name COLUMN]
[--copy-source-column]
[--flatten-path]
[--flatten-delimiter FLATTEN_DELIMITER]Required Parameters
| Kafka bucket name containing the source topic. |
| Source Kafka topic name. |
| Target database table name for expanded data. |
| Column definitions as a comma-separate list of name-type pairs. These are the fields (and their type) in the Kafka event source that are parsed. They are parsed into columns of the same name and type in the target table. Columns are created in the target database table if they do not already exist. Example This selects fields |
Options
| Tenant ID (optional) for the schema. |
| Source column name (only 'value' is supported) |
| Target table schema (optional) |
| The format of the source column, that is expanded (only 'json' is supported) |
| If set, the raw source event message (in binary JSON format) is copied to the target table. |
| Flatten nested JSON paths. If set, nested JSON paths are parsed as a flat path. Example: this JSON block: is parsed to user_profile_name, user_profile_age (where the delimiter is '_'). |
| Delimiter for flattened paths (default: '__') |
Example
This example shows the blobexpansion create command with sample values.
vcli: admin> blobexpansion create --tenant-id 1 --database-name kafka_bucket --table-name my_topic --source-column-name value --target-table-name expanded_data --target-table-schema public --expansion-format json --arrow-schema col1-string,col2-int,col3-float --copy-source-column --flatten-path --flatten-delimiter __