System Architecture

Shaper is a single binary that can run without any external dependencies. It has low system requirements and can be deployed in various environments.

Components

1. Shaper is written in Go with an embedded React UI.
2. Internal state (users, dashboards, etc.) is stored in SQLite.
3. Shaper uses NATS internally to coordinate state which allows running Shaper as a multi-node cluster (see below).
4. Optionally, data can be replicated to an S3-compatible object storage.
5. And of course Shaper uses DuckDB as query engine and data store.

Data Storage

Instead of relying on a separate database such as Postgres or MySQL, Shaper allows you to connect to an S3-compatible object storage to keep your data safe. Object storage is cheaper and simpler than requiring a separate database.

Clustering

Clustering Shaper allows for high availability deployments and enables you to scale read performance beyond a single node.

Shaper uses NATS to replicate internal state and data ingested via the HTTP and NATS APIs across all nodes. NATS also coordinates tasks scheduling across all nodes.

Shaper has a built-in NATS server, which can be clustered. Alternatively, you can also use an external NATS server.

Flexible Deployment Options

• Shaper can be run on your local machine without any configuration required.
• You can easily deploy Shaper to one or more virtual machines depending on your reliability and performance needs.
• Shaper also runs on stateless container platforms such as AWS ECS, Google Cloud Run, Docker Swarm or any Kubernetes host.

Non-Goals of Clustering

Shaper does not partition data across nodes.

While you can store data in Shaper’s internal DuckDB, it’s not intended to replace existing data storage infrastructure.

If you are working with large datasets or high write throughput, you need separate data infrastructure such as a data warehouse or data lake plus data processing tools that fit your needs. Use Shaper to analyze and visualize data, not to store it.