Managing Alarms and Events

Viewing Events

To view events in VAST Web UI, select Alarms & Events and then Events to open the Events page.

Note
The event log holds the latest up to 1.5 million events.

For each event, the following information can be displayed:

Tip
To display or hide fields, click to the right of a column title, and then click to open a dropdown where you can select or unselect fields.

ID	The ID of the event.
Date	The date of the event.
Object Name	The name of the object involved in the event.
Message	A description of the event.
Event Type	The type of event that occurred. See Event Types.
Object Type	The type of the object involved in the event.
Origin	The initiator of the event. Possible values: Cluster Management Security User
Severity	The severity of the alarm triggered by the event. This setting is inherited from the event definition, and you can modify it. Possible values: Info. No alarm is raised. Minor. An alarm of minor severity is raised. Major. An alarm of major severity is raised. Critical. An alarm of critical severity is raised.

Aggregating Similar Events and Alarms

Alarms or events that are similar (but may originate in different objects of the same type) are aggregated, and appear as a single entry in the Alarms or Events tables. When alarms are aggregated, the Object Name shows the Cluster on which the alarm occurred. The message text will list the actual objects on which the event occurred.

For example, the following record shows an event that occurred a two CNodes:

vast17-kfs ALERT[P166625:E1:S255:Fa35e8 time="2024-05-07 23:01:13.049879590"]: A group with gid=1305910462 and name=vast-test-group-1305910462-0 and already exist in the map (new name=)vast-test-group-1305910462-1 counter: 7776. Occurred on cnode-3-9 (172.16.3.9) [v17cn1], cnode-3-8 (172.16.3.8) [v17cn2]

Note
Events and alarms are aggregated for ALARM and PANIC events only.

Filtering Hardware Events for Lower-Level Components

Hardware alarms and events for lower-level components (for example an SSD unit or a fan) are suppressed if they also trigger events for higher-level components (for example the CNode or DNode containing them). In this case, only the events for the higher-level component appear in the Events or Alarms table. So, for example, it alarms are generated for an SSD unit and the CNode containing it, the alarm for the SSD is suppressed, and only the alarm for the CNode appears.

Handling Alarms

Alarms are classified with three levels of severity: minor, major, and critical. The severity of an alarm is inherited from the definition of the event that triggered the alarm.

Alarms are automatically removed when the events that triggered them are resolved. In many cases, automatic recovery features handle and resolve events. Occasionally, user intervention may be required to resolve the issue.

How do I know when there are active alarms?

When one or more alarms are present, an indicator shows the number of alarms present:

How can I see what is causing alarms?

To see a summary of alarms, classified by severity, click the alarm indicator:

To read the alarm messages in more detail, open the Alarms page:

To open the Alarms page that lists all the alarms, click See All.

To open the Alarms page that shows alarms of a particular severity, click more > at the bottom of the severity pane.

To open the Alarms page from the left navigation menu at any time, select Alarms & Events and then Alarms.

Can I Acknowledge Alarms?

If you've seen the alarm messages and you want to hide them from view, you can hide all the current alarms from view by clicking the Acknowledge all button (). The alarms remain visible to the root and support users.

Default Notification Actions

You can set up default notification actions so that VAST Cluster can send notifications of events to configured destinations. Default notification actions apply to all event definitions on the VAST Cluster cluster and can be overridden per event definition.

The following default notification actions can be configured:

Sending notifications to email recipients. This action requires that an outgoing SMTP server is configured.
Triggering a webhook that sends alarms to an external applications using HTTP requests.
Sending messages to a Linux syslog server.

Default notification actions can be disabled and then re-enabled during maintenance activities.

If you want to disable notification actions or to create a custom action for a particular event definition, modify the event definition as described in Modifying Event Definitions. Custom action settings include webhooks and syslog settings, but do not include the outgoing SMTP server setup. The outgoing SMTP server is global, required for default email recipients as well as for any alternate email recipients that you define per event definition.

Configuring Default Notification Actions

From the left navigation menu, select Settings and then Notifications.

Under SMTP Setup, complete the fields to configure SMTP for sending email notifications:

SMTP Host	The host name of the SMTP server. For example: mail.company.com.
SMTP Port	The port used by the SMTP server to send outgoing emails. The most commonly used port for SMTP is port 25, although some IPs deny its use in order to block spam. SMTP servers often support alternate ports, including port 587.
SMTP User	User for SMTP host authentication.
SMTP Password	The password for the SMTP user.
Use TLS	Enable this setting to send emails over a TLS connection.

Under Email Setup, complete the fields to set up email message properties and recipients:

Email Sender

The sender email address that is included in outgoing emails. This setting applies to all alarm notification emails.

Example: do_not_reply@company.com

Email Subject

The email subject to be be included in outgoing emails. This optional setting applies to all alarm notification emails.

Example: VAST Alarm

If you want VAST Cluster to include the alarm description as the email subject, leave this field blank.

Email Recipients

Default email recipients. These recipients receive notifications of all alarms except those triggered by events that have a different list of email recipients specified in the event definition or for which default notification actions are disabled.

Enter as a comma-separated list of email addresses (no spaces).

Example: storage_admin@company.com,bsmith@company.com,abrown@company.com

Under Webhook Setup , optionally set custom names for event severity levels used when sending webhook payloads (in vastdata_severity). In each of the fields provided, overwrite the default value with your own text.

To configure sending alarm information to a syslog server, select Syslog Setup and complete the fields:

Note
For server-side configuration for the syslog server, see Sending Alarms to a Syslog Server.

Syslog Host	Specify the syslog server's IP address.
Syslog Port	Specify the port number that the server listens on for syslog requests. Default: 514
Syslog Protocol	Specify either of the protocols for communicating with the remote syslog server: TCP UDP (default) The protocol you choose must be enabled on the syslog server. See Sending Alarms to a Syslog Server.
Enable Secure Log Audit	Toggle on (default) or off to enable or disable logging to the `/var/logs/secure` log.
Enable VMS Audit	Toggle on (default) or off to enable or disable auditing of VMS operations.
Enable Shell Audit	Toggle on or off (default) to enable or disable auditing of CNode and DNode shell commands.
Enable IPMI Audit	Toggle on or off (default) to enable or disable auditing of CNode and DNode IPMI commands.
Audit Logs Retention	Enter the number of days to store audit logs on the syslog server.

Click Save.

Disabling and Re-enabling Default Notification Actions

When running maintenance tasks, you may need to disable default notification actions. To do so, click the Disable Actions button at the bottom of the dialog.

The button is a toggle button. Click it to disable the configured actions. The default notification actions are disabled and the button text changes to Enable Actions. Click it again to re-enable the default notification actions.

Viewing Event Definitions

From the left navigation menu, select Alarms & Events and then Event Definitions to open the Event Definitions page.

The following information can be displayed for each event definition:

Tip
To display or hide fields, click to the right of a column title, and then click to open a dropdown where you can select or unselect fields.

ID	The ID of the event definition.
Object Type	The type of objects being monitored for the event. This could be the cluster, a hardware component such as a NIC, or a logical entity such as a snapshot.
Property	The monitored property of the object. For example: state, number of PCI errors, percentage memory usage.
Event Type	Indicates the type of change in the property value that triggers an alarm. See Event Types
Severity	The severity of the alarm triggered by this event. Can be critical, major, minor or N/A. If N/A , no alarm is raised.
Enabled	Indicates Yes if the event definition is enabled (default), or No if it was disabled.

Modifying Event Definitions

From the left navigation menu, select Alarms & Events and then Event Definitions.
In the Event Definitions page that opens, find the event definition you want to modify and note its event type (Object Modified, Threshold or Rate).
Right-click the event definition that you want to modify, and select Edit.
The Event Definition dialog appears.

Under Event, review and change the settings depending on the event type:

For events of the Object Modified type:

Severity	Determine the severity of the alarm that is raised when this event occurs.
Trigger On	Specify the value or the monitored property at which to trigger an alarm.
Trigger Off	Specify the value or the monitored property at which to trigger off an alarm.

For events of the Threshold type:

Severity	Determine the severity of the alarm that is raised when this event occurs.
Operator	Select the operator to be used when comparing the monitored property value with the threshold: greater than. The alarm is triggered when the monitored property increases to higher than the specified threshold value. greater equal. The alarm is triggered when the monitored property increases to the specified threshold value or higher. lower than. The alarm is triggered when the monitored property falls below the specified threshold value. lower equal. The alarm is triggered when the monitored property falls to the specified threshold value or below it.
Threshold	Specify the threshold value of the monitored property.

For events of the Rate type:

Severity	Determine the severity of the alarm that is raised when this event occurs.
Operator	Select the operator to be used when comparing the monitored property value with the threshold: greater than. The alarm is triggered when the monitored property increases to higher than the specified threshold value. greater equal. The alarm is triggered when the monitored property increases to the specified threshold value or higher. lower than. The alarm is triggered when the monitored property falls below the specified threshold value. lower equal. The alarm is triggered when the monitored property falls to the specified threshold value or below it.
Threshold	Specify the threshold value of the monitored property.
Timeframe	Define the period of time during which to monitor the property. Enter a number followed by a time unit without spaces. Valid time unit designations: `s` - seconds `m` - minutes `h` - hours `D` - days `M` - months For example: 1h

Under Action, optionally select the actions that are performed when the event is triggered. You can choose to send a notification to an external application using a webhook, and/or send email notifications to selected recipients.
- To send a notification to specific external applications using webhooks, select the webhooks for these applications from the Select Webhook dropdown. If none are selected, notifications are sent to all enabled webhooks.
  Note
  Define webhooks in the Alarms & Events → Webhooks tab. See Managing Webhooks for more details.
- To send an email notification, enter a comma-separated list of email addresses in Email Recipients.
  Note
  An SMTP server and sender email address must be defined in the SMTP Setup tab of the Notifications section of the > Settings page. See Default Notification Actions.
- To disable the above actions, slide Disable actions to the right.
- To perform the actions on in the case of an alarm, slide Alarm only to the right. Otherwise all events for this definition will trigger the actions.
  For example, if the object type is CNode, and the Property is state , then if Alarm Only is disabled, any change in state of a CNode will trigger the actions you selected for this event.
Click Save to save the changes or click Save & Test to save your changes and send a test notification.

Enabling and Disabling Event Definitions

By default, an event definition is enabled. If you disabled an event definition, VAST Cluster does not perform notification actions (such as sending an email or logging a message to a syslog server) in case the event occurs on the cluster. You can re-enable a disabled event definition to resume notification actions for this event.

The current enabled/disabled status for each event definition is displayed in the Enabled column in the Event Definitions page.

To enable or disable an event definition:

From the left navigation menu, select Alarms & Events and then Event Definitions.
Open the Actions menu for the event definition you want to enable or disable, and then select Activate to enable or Deactivate to disable it.
Click Yes to confirm your action.
The event definition is enabled/disabled. After a few moments, you can see the changed state displayed in the Enabled column in the Event Definitions page.

Downloading Alarms and Events

When you're viewing the event log in the Events page or the current alarms in the Alarms page, you can download all of the events in the log or all of the current alarms.

To download a complete log or a complete list of the current alarms, click the download button at the top right of the list: .

Documentation Index