Skip to content
karldreherPublic

About

Minimally viable KV storage for local use

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

stow

A small, local, in-memory key-value server.

This service is meant for process coordination in local and isolated environments. It does not suggest nor intend to have the sophistication of etcd or similar projects.

What problem does it solve?

This is especially build for inter-Agent communication on a local device (such as a laptop) or container environments that are isolated. It is especially built for parallel agents that need to agree on state.

This is a platform primitive designed for local use, and intended to be used in flexible ways that can adapt to many different processes that could work in concert with one another.

Wait. Why would agents need to agree on state?

There are many reasons.

Many things can be virtualized on a computer or used in ways that minimize concurrency concerns, like

  • CPUs
  • Networking
  • Databases and instances

While this project does not intend to argue about the semantics or potential workarounds, some resources have inherently limited capacity or cannot meaningfully be shared simultaneously:

  • An entire GPU's capacity.
  • All the RAM on your computer.
  • A monitor that shows information to a user (While multiple windows have been in computing for more than 30 years, the number of eyes available to humans remains bounded, as well as attention span)
  • Audio output.
  • Taking control of the mouse. (Hmmm...)
  • Any process or series of processes that implies order of operations

Wait. Why can't i just ask my agent to do it for me?

Be my guest.

While there are other ways, including writing to a local filesystem, this project's goal is to provide a highly performant, minimal service that may be used in place of token-implicit methodology when there are simpler, solution-oriented ways to get to the same result with better performance. This unlocks the ability for agents and other parallel processes to scale, without the waste associated with tooling churn.

Install

Download the archive for your platform (macOS and Linux, arm64 and x86_64) from the releases page. Extract the archive and put the stow binary somewhere on your PATH.

With a Rust toolchain you can build from source instead:

cargo install --git https://github.com/karldreher/stow

Quick start

Start the server. It listens on 127.0.0.1:7319 by default:

stow serve

Then store, read, list, and delete a value:

curl -X PUT --data-binary 'hello' http://127.0.0.1:7319/api/v1/set/greeting
curl http://127.0.0.1:7319/api/v1/get/greeting
curl http://127.0.0.1:7319/api/v1/keys
curl -X DELETE http://127.0.0.1:7319/api/v1/delete/greeting

The same binary is also a client, so you can skip curl:

stow set greeting hello
stow get greeting
stow keys
stow del greeting

Client commands talk to 127.0.0.1:7319 by default. Use --addr HOST:PORT or the STOW_ADDR environment variable to reach another instance.

Waiting

stow get --wait-for blocks until a key reaches a state you describe. stow lock --wait blocks until you hold a mutex. They do different jobs:

  • get --wait-for only observes a key and never changes it, so it cannot acquire anything. Use it to wait on state you do not plan to claim, such as a published value, a task status, or a lock someone else releases.
  • lock and lock --wait are the only way to take a mutex, because the check and the write are one atomic step.
  • get --wait-for absent && stow lock ... is racy: every waiter wakes at once and the losers get a conflict. Use lock --wait instead.
stow lock lock/build --ttl 60 --wait
stow get build-sha --wait-for present
stow get task-42/status --wait-for value=done --timeout 300

Waiting is polling with back-off, so the server stays simple and small. See USAGE.md for conditions, timeouts, and exit codes.

Data lives in memory only and is lost when the server stops, unless you pass --save-file <PATH> to restore it on the next start (see USAGE.md).

For more examples, including conditional writes, expiring leases, and the full API reference, see USAGE.md.

Memory

stow has the capability to run many instances side by side, so it keeps memory small. The key table shrinks after large deletes and expiry sweeps.

Measured on macOS arm64 with a release build (RSS from ps):

State RSS
Idle about 2.5 MB
10,000 keys with 1 KiB values about 14 MB

About

Minimally viable KV storage for local use

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages