The VAST Database includes a VAST Query Engine. This query engine provides improved performance for database operations.
VAST Database clients can access the query engine using a VAST ADBC client library.
VAST Query Engine Features
The VAST Query Engine supports the following:
Vector search. The ability to query vectors in the database for the nearest neighbors.
Column Allow/Deny. The ability to restrict access to columns and rows in a database based on a users identity.
Filter pushdown. This is an optimized scan/query of selected rows and/or columns in a database table.
Nested data types. The ability to nest datatypes such as struct and array.
Accessing VAST DataBases using the Query Engine
In order to access the VAST DataBase using the Query Engine, you will need the following:
A virtual IP pool dedicated to the Query Engine (see Creating Virtual IP Pools for the VAST Query Engine)
An S3 user with permission to access the DataBase, and with an access and secret key pair (the keys are used by the client application). See Managing Local Users.
An identity or bucket policy for the user, that sets the permissions to access the database. See Creating Identity Policies .
A client application that uses the VAST Query Engine ADBC library to access the Query Engine (see VAST DataBase Query Engine ADBC Driver for Client Applications).
Creating Virtual IP Pools for the VAST Query Engine
The Query Engine requires a dedicated virtual IP pool for client access. Create this pool with the role Query Engine, using the VAST Web UI.
Navigate to the Network Access page, and select Virtual IP Pools.
Click Create Virtual IP Pool.
Compete details for the virtual IP pool following the procedure described in Creating Virtual IP Pools, selecting role type Query Engine. Define also a FQDN in the DNS for the virtual IP pool (see Step 6).
VAST DataBase Query Engine ADBC Driver for Client Applications
VAST provides an open-source ADBC driver that you can use to provide access for your client application to the VAST Query Engine.
Download the driver from https://github.com/vast-data/vastdb-adbc-driver.
The ADBC can be used in a Linux x86_64 environment, for programming languages that have a driver manager (for example, C++, Rust, Python, Ruby, Go, and Java).
When using this driver, you will need to provide a virtual IP address and an access/secret key pair, to access databases (see Creating Virtual IP Pools for the VAST Query Engine.
NOTE: The ADBC driver can be used with VAST Cluster 5.4.0 or later.
ADBC User Impersonation
Starting with VAST Cluster 5.4.1, you can run VAST ADBC driver commands with the credentials of the end user (instead of using the service account of the VAST ADBC driver). To do so:
Grant the
s3:EndUserImpersonationpermission to the VAST ADBC driver's service account through the identity or bucket policy. For example:{ "Version": "2012-10-17", "Statement": [ { "Sid": "Allow end user impersonation on Tabular operations", "Effect": "Allow", "Action": "s3:EndUserImpersonation", "Resource": ["arn:aws:s3:::*"] } ] }Use the vast.db.end_user connection option to pass the username of the end user along with the query. For example:
conn = adbc_driver_manager.dbapi.connect( driver="driver_path", db_kwargs={ "vast.db.endpoint": "server_ip", "vast.db.access_key": "access_key", "vast.db.secret_key": "secret_key", }, conn_kwargs={ "vast.db.end_user": "user_name", } )