Logs and Traces

Generating Logs

By default, the runtime captures stdout (such as print() statements) and logs from standard Python loggers, automatically transmitting them via OpenTelemetry to VAST DataEngine.

The runtime also provides an embedded logger, which is a standard pythonic OpenTelemetry logger:

ctx.logger

You can use it to generate telemetry logs. Logs generated by the embedded logger in the ctx object can be visualized by the DataEngine API.

In the following example, several logs are set to info level severity:

from vast.dataengine.sdk import VastEvent
import os


def init(ctx):
    pass


def handler(ctx, event: VastEvent):
    ctx.logger.info(f"Event id: {event.id}, timestamp: {event.timestamp}")
    ctx.logger.info(f"Extensions: {event.extensions}")
    data = event.get_data()
    string_format = os.environ.get("STRING_FORMAT", "")
    ctx.logger.info(f"CloudEvent data is: {data}")
    ctx.logger.info(f"CloudEvent event is: {event}")

    # 1. Retrieve the outer secret container block by name (e.g., "secrets")
    secret_dict = ctx.secrets.get("secrets", {})

    # 2. Unpack the inner key-value dictionary into the format string.
    # Placeholders in STRING_FORMAT (e.g., "Hello {DB_USER}") must match the exact secret key names.
    message = string_format.format(**secret_dict)

    # Avoid logging raw secret strings to telemetry logs for security
    ctx.logger.info("Successfully formatted response message with secrets.")
    return message

Generating Traces and Spans

The embedded tracer is a standard OpenTelemetry tracer. You can generate telemetry spans using:

Manual Spans: Direct context manager calls via ctx.tracer.start_as_current_span() or ctx.start_as_current_event_span().
Decorator-Based Spans: Decorating functions with @span_wrapped or @event_span_wrapped.

Traces and spans generated by the SDK are automatically visualized in the DataEngine dashboard.

Manual Spans

You can create fine-grained spans within code blocks using:

ctx.tracer.start_as_current_span()
- Mechanism: Passes execution directly through to standard OpenTelemetry (self._user_tracer.start_as_current_span), creating a span tied to the local active trace context.
- When to use: For generic internal operations, helper functions, database calls, or payload transformations that occur within the function execution flow without needing explicit linkage to an incoming event's distributed trace.
ctx.start_as_current_event_span()
- Mechanism: Requires a VastEvent object (event: VastEvent) and invokes a runtime helper (_create_event_span_fn) to extract trace headers/context from that specific event and attach the new span directly to the event's parent trace. It also injects pipeline metadata (self._pipeline_rev_id) and falls back to a standard span if the runtime context is unconfigured.
- When to use: At the entry point of event handling logic or when processing a specific incoming event, ensuring distributed tracing continuity between the upstream event publisher and your function execution in DataEngine.

Example using ctx.tracer.start_as_current_span():

def handle_event(ctx, event):
    # Top-level span for the handler
    with ctx.tracer.start_as_current_span("Video Reasoner Handler") as handler_span:
        handler_span.set_attribute("event_type", "example")

        # Nested span for event parsing
        with ctx.tracer.start_as_current_span("Event Parsing") as parse_span:
            data = event.get_data()
            event_info = parse_s3_event(data)

            parse_span.set_attributes({
                "bucket": event_info.get("bucket", "unknown"),
                "key": event_info.get("key", "unknown"),
            })

        # Nested span for payload processing (generic operation)
        with ctx.tracer.start_as_current_span("Payload Processing") as process_span:
            # Perform processing on local data instead of making external API calls
            payload_size = len(str(data))

            # Record metrics directly to the span
            process_span.set_attribute("payload_size_bytes", payload_size)
            process_span.set_attribute("status", "success")

Decorator-Based Spans

Decorators provide a clean, declarative way to trace functions without manually managing with blocks or ctx.tracer references.

@event_span_wrapped automatically attaches span tracing to a VastEvent within event handlers:

from vast.dataengine.sdk import event_span_wrapped, VastEvent

@event_span_wrapped("Video Analysis Handler", attributes={"pipeline": "video_v1"})
def handler(ctx, event: VastEvent):
    ctx.logger.info(f"Processing event {event.id}")
    data = event.get_data()
    return process_payload(data)

@span_wrapped wraps helper functions and creates a span using either a custom name or the function's name:

from vast.dataengine.sdk import span_wrapped

@span_wrapped("Payload Transformation")
def process_payload(data):
    # Code executed here is automatically timed and traced in OpenTelemetry
    return str(data).upper()

Documentation Index

Logs and Traces

Generating Logs

Generating Traces and Spans

Manual Spans

Decorator-Based Spans