Timeseries Configuration

Timeseries is configured through CLI arguments and a Prometheus-compatible YAML configuration file. The CLI arguments control server startup, while the YAML file defines scrape targets, storage backends, and operational parameters.

CLI Arguments

Pass these flags when starting the opendata-timeseries binary.

Flag	Environment Variable	Default	Description
`-c`, `--config`	`PROMETHEUS_CONFIG_FILE`	none	Path to the `prometheus.yaml` configuration file
`-p`, `--port`	`OPEN_TSDB_PORT`	`9090`	Port to listen on

./opendata-timeseries --config prometheus.yaml --port 9090

Configuration File

The configuration file follows the Prometheus YAML format with additional sections for storage. The generic placeholders used in this reference are:

<string>: a regular string
<int>: an integer value
<duration>: a duration matching [0-9]+(ms|s|m|h|d) (e.g. 30s, 1m, 2h)

A value enclosed in [ ] is optional. The default value is shown after | default =.

# Default settings applied to all scrape jobs.
global:
  # How often to scrape targets.
  [ scrape_interval: <duration> | default = 15s ]

# List of scrape job configurations.
scrape_configs:
  [ - <scrape_config> ... ]

# Storage backend configuration (required).
storage:
  <storage_config>

# How often (in seconds) to flush data from memory to durable storage.
[ flush_interval_secs: <int> | default = 5 ]

# Startup cache warmer (enabled by default).
[ cache_warmer: <cache_warmer_config> ]

# Durable buffer consumer. See the Stateless Ingest page for the full write path.
[ buffer_consumer: <buffer_consumer_config> ]

`<scrape_config>`

Each scrape config defines a set of targets and the parameters for scraping them.

# Name for this scrape job. Added as the `job` label on all scraped metrics.
job_name: <string>

# Override the global scrape interval for this job.
[ scrape_interval: <duration> | default = <global.scrape_interval> ]

# List of static target groups to scrape.
static_configs:
  [ - <static_config> ... ]

`<static_config>`

A static config defines a list of targets and a common set of labels to apply.

# Endpoints to scrape in host:port format.
targets:
  [ - <string> ... ]

# Labels to attach to all metrics scraped from these targets.
labels:
  [ <string>: <string> ... ]

`<storage_config>`

Storage is tagged by type.

# The storage backend type.
type: <string> # One of: InMemory, SlateDb

InMemory
SlateDb

Stores data in memory only. Data is lost on restart. Useful for testing and development. No additional fields are required.

storage:
  type: InMemory

Persists data to an object store via SlateDB.

storage:
  type: SlateDb

  # Path prefix for SlateDB data within the object store.
  [ path: <string> | default = "data" ]

  # Path to a SlateDB settings file (TOML/YAML/JSON).
  # If omitted, SlateDB checks for SlateDb.toml, SlateDb.json, or SlateDb.yaml
  # in the working directory and merges any SLATEDB_-prefixed environment variables.
  [ settings_path: <string> ]

  # Cache for SST data blocks (optional). See <cache_config> below.
  [ block_cache: <cache_config> ]

  # Cache for SST metadata: filters, indexes, and stats blocks.
  # Set this whenever block_cache is set. See <cache_config> below.
  [ meta_cache: <cache_config> ]

  # Object store provider configuration (required).
  object_store:
    <object_store_config>

`<cache_config>`

Two slots share this cache configuration: block_cache holds decoded SST data blocks, and meta_cache holds SST metadata (filters, indexes, and stats blocks). When either is set, SlateDB routes data blocks to block_cache and metadata blocks to meta_cache. A slot left unset caches nothing for that class of block.

Setting block_cache without meta_cache leaves filters and indexes uncached, so every read fetches them from the object store again. Because each read consults the filter of every SST that might hold the key, query latency then grows with SST count. Always pair block_cache with a meta_cache.

A cache is tagged by type: FoyerHybrid (memory + disk tiers) or FoyerMemory (memory only), both backed by foyer.

FoyerHybrid
FoyerMemory

Two tiers: in-memory plus on-disk (ideally NVMe). Suited to the block_cache slot, where capacity matters more than the last few microseconds of latency.

block_cache:
  type: FoyerHybrid

  # In-memory tier capacity in bytes.
  memory_capacity: <int>

  # On-disk tier capacity in bytes.
  disk_capacity: <int>

  # Directory for the on-disk tier.
  disk_path: <string>

  # When entries are promoted to the disk tier.
  # WriteOnInsertion persists every cached block; WriteOnEviction
  # only persists on memory eviction.
  [ write_policy: <string> | default = WriteOnInsertion ]

  # Flush thread count for the disk engine.
  [ flushers: <int> | default = 4 ]

  # Flush-pipeline buffer pool size in bytes. Actual allocation is ~2x
  # this value (flushers double-buffer). Defaults to memory_capacity / 32.
  [ buffer_pool_size: <int> ]

  # Write-queue size threshold in bytes. Entries beyond this are dropped
  # rather than blocking the cache.
  [ submit_queue_size_threshold: <int> | default = 1073741824 ]

A single in-memory tier. Suited to the meta_cache slot: metadata blocks are small and every read consults them, so they belong in memory, and the cache warmer rebuilds the cache cheaply after a restart. Size it to hold the full live filter and index set so it never evicts.

meta_cache:
  type: FoyerMemory

  # Cache capacity in bytes.
  capacity: <int>

  # Number of shards. When absent, foyer derives a default from the
  # available CPU count.
  [ shards: <int> ]

See the production guide for recommended sizes.

`<object_store_config>`

The object store is tagged by type.

Local filesystem
AWS S3
In-memory

object_store:
  type: Local
  # Directory path for local storage.
  path: <string>

object_store:
  type: Aws
  # AWS region (e.g. us-west-2).
  region: <string>
  # S3 bucket name.
  bucket: <string>

object_store:
  type: InMemory

`<cache_warmer_config>`

On startup, the warmer discovers the time buckets that overlap the warm window and hands the SSTs backing them to SlateDB’s cache manager, which loads each SST’s filters and index (plus its sample data blocks when include_samples is set) directly into the configured caches. Filters and indexes land in meta_cache, data blocks in block_cache. The first queries after a restart then run against warm caches instead of paying object-store round trips. The warmer runs once and exits. It is a no-op when neither cache is configured. Pairing it with an in-memory meta_cache works well: the cache loses its contents on restart, and the warmer rebuilds them in seconds. Enabled by default. Set cache_warmer: null to disable.

cache_warmer:
  # How far back to warm on startup.
  [ warm_range: <duration> | default = 24h ]

  # Whether to warm sample data blocks in addition to filters and indexes.
  [ include_samples: <bool> | default = true ]

Match warm_range to the time range your dashboards and alerts query most. A warm window that exceeds cache capacity wastes work: the cache evicts some warmed blocks before a query ever reads them.

`<buffer_consumer_config>`

Enables the durable-queue write path. See the Stateless Ingest page for the full architecture, the OpenTelemetry Collector side, and operational guidance.

buffer_consumer:
  # Object store holding the buffer queue. May differ from the TSDB's
  # storage bucket.
  object_store:
    <object_store_config>

  # Must match the manifest path configured on the producer side.
  [ manifest_path: <string> | default = "ingest/manifest" ]

  # Delay between polls when the queue is empty.
  [ poll_interval: <duration> | default = 1s ]

Requires read-write mode. When absent, no consumer starts.

Examples

Local development
Production (S3)
Testing (in-memory)

prometheus.yaml

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: "my-app"
    static_configs:
      - targets: ["localhost:9090"]

storage:
  type: SlateDb
  path: data
  object_store:
    type: Local
    path: ./data

flush_interval_secs: 5

prometheus.yaml

global:
  scrape_interval: 30s

scrape_configs:
  - job_name: "node-exporter"
    scrape_interval: 15s
    static_configs:
      - targets: ["host-a:9100", "host-b:9100"]
        labels:
          env: production
  - job_name: "api-server"
    static_configs:
      - targets: ["api-1:8080", "api-2:8080"]
        labels:
          env: production

storage:
  type: SlateDb
  path: data
  object_store:
    type: Aws
    region: us-west-2
    bucket: my-metrics-bucket

prometheus.yaml

global:
  scrape_interval: 5s

scrape_configs:
  - job_name: "test-app"
    static_configs:
      - targets: ["localhost:9090"]

storage:
  type: InMemory

Concepts

Timeseries

Log

Vector

Buffer

Timeseries Configuration

CLI Arguments

Configuration File

`<scrape_config>`

`<static_config>`

`<storage_config>`

`<cache_config>`

`<object_store_config>`

`<cache_warmer_config>`

`<buffer_consumer_config>`

Examples

​CLI Arguments

​Configuration File

​<scrape_config>

​<static_config>

​<storage_config>

​<cache_config>

​<object_store_config>

​<cache_warmer_config>

​<buffer_consumer_config>

​Examples

CLI Arguments

Configuration File

`<scrape_config>`

`<static_config>`

`<storage_config>`

`<cache_config>`

`<object_store_config>`

`<cache_warmer_config>`

`<buffer_consumer_config>`

Examples