You create, view, modify and delete event topics similar to VAST Database tables. When you create a VAST Event Broker view, VAST Cluster automatically creates an associated VAST database with a schema labelled Kafka-Compatible Broker Topics. Under this schema, you can create and manage topics where events will be recorded.
Creating a Topic in VAST Web UI
To create a topic in VAST Web UI:
In the left navigation menu, choose DataBase -> VAST Database to open a list of databases available on the cluster.
Click an arrow on the left to the database named after your Kafka-enabled view and then click Kafka-Compatible Broker Topics under the database name.
Click the Add Topic button on the right to create a new topic.
In the Create Topic dialog, complete the following required fields:
Topic Name
Enter a name for the topic.
Number of partitions
Enter the number of partitions for the topic.
Each partition can hold up to 1000 topics. The number of partitions in a topic cannot be changed after the topic has been created.
Note
The number of partitions can be set only when creating a topic. It cannot be modified by editing an existing topic.
Under Data Preservation and Efficiency:
Retention period
Specify the amount of time to keep an event record in the topic. When the retention period for a record expires, the records is deleted from the topic.
The default retention period is seven days. The minimum allowed retention period is 6 hours. A value of -1 means no retention period.
Compaction
If this flag is set, VAST Cluster keeps only the latest record version for each key in the partition log. Any previous versions of the record are deleted.
Note
Compaction is performed asynchronously in the background. At some point in time, duplicate keys may exist.
Under Advanced Options -> Timestamp source:
Time event is created (producer)
Sets the event timestamp based on the time when the event was encountered at the event producer.
Time event is logged (server)
Sets the event timestamp based on the time when the event record was added to the log at the event broker.
Under Advanced Options -> Event time validation period:
Infinity
For Time event is created (producer) timestamps, specifies that any difference between the producer time and the server time is acceptable, e.g. the message timestamp can be earlier or later than the broker timestamp for any amount of time.
Define Manually
For Time event is created (producer) timestamps, specifies the acceptable difference between the producer time and the server time:
Not before: Determines how much earlier the message timestamp can be than the broker timestamp. If this value is exceeded, the message is rejected.
Not after: Determine how much later the message timestamp can be than the broker timestamp. If this value is exceeded, the message is rejected.
Click Create.
Modifying a Topic in VAST Web UI
To modify a topic in VAST Web UI:
In the left navigation menu, choose DataBase -> VAST Database to open a list of databases available on the cluster.
Click an arrow on the left to the database named after your Kafka-enabled view and then click Kafka-Compatible Broker Topics under the database name.
Click on the the topic you want to edit so that it is displayed in the central pane. Then click the Edit button on the top right.
In the Create Topic dialog, make changes as needed. See Creating a Topic in VAST Web UI for a description of each field.
Click Create.
Managing Topics in VAST CLI
Use the following VAST CLI commands to manage event topics:
Task | Command |
|---|---|
Create a topic | |
Modify a topic | |
Delete a topic | |
List topics | |
View a topic |