Skip to content
Taleshape LogoTaleshape LogoTaleshape
Blog

Blog

Claude Dashboards vs Shaper

- 2 min read
Jorin Vogel
Jorin Vogel

Shaper is an open source dashboarding product. Now Anthropic released Claude Dashboards. What does that mean for Shaper?

The last days everyone on Data Analytics LinkedIn has been talking about how Claude Dashboards just killed the need for BI software.

This whole discussion only partially affects Shaper though: The Business Intelligence market has been crowded even two years ago when I started building Shaper.

Shaper never tried to compete with traditional BI tooling:

The problem Shaper solves:

  1. You want to share dashboards and reports with customers,
  2. but you are working with sensitive personal data that must stay in-house,
  3. and you need to be able to verify and audit results for correctness.

That’s why Shaper:

  1. focuses on embedded analytics,
  2. is fully open source and runs in your infrastructure,
  3. and defines dashboards as SQL code you store in Git.

None of these problems are solved by Claude Dashboards or the tools that came before it. I don’t expect that any healthcare company or finance org I work with to be willing to send PII to Claude or share AI generated dashboards with customers without a review process to approve the underlying analytics. Claude Dashboards looks like a great tool to query less critical data and to prototype dashboards before letting Claude translate them into Shaper SQL.

If you like to Shaper and want the ease of Claude to generate dashboards, checkout how to use our AI Agent Workflow to generate dashboards with Claude Code.

Why we open sourced Single Sign-On

- 3 min read
Jorin Vogel
Jorin Vogel

We just added Single Sign-On functionality to Shaper. And we made it open source.

Single Sign-On (SSO) is the most obvious feature to separate from the main product and sell at a premium to enterprise clients.

You don’t need it to start using the product, but any bigger organization cannot do without it.

Why is SSO crucial to bigger organizations?

Section titled “Why is SSO crucial to bigger organizations?”

Obviously it’s not because large organizations care so much about UX and cannot live without the convenience of not having to type a password. Instead, it’s all about security: No weak passwords. Centralized access rights. Audit logging.

Where does this leave smaller companies?

Section titled “Where does this leave smaller companies?”

Should they care less about security? Is security something to build later on after you are already successful? I and many others don’t think so. That’s why there are websites like https://sso.tax/.

A Compromise

Section titled “A Compromise”

We decided to make SSO part of open source Shaper. But with a nuance:

JWT-based SSO is completely open source. You implement a single endpoint in your application and get complete control over how the user is authenticated. This is the simplest way to integrate Shaper into your existing application.

In addition to this, we provide OpenID Connect integration (OICD) and Proxy Header Auth support as part of our commercial offer. If you have an OICD provider or an internal API gateway integrated in your infrastructure, then integrating with them requires zero custom code. But investing in this infrastructure just for Shaper is definitely more complex than creating a JWT authentication endpoint.

Please reach out if you have any feedback or if you are looking for support with implementing SSO.

Build Your Own Data Analytics Slack Agent With Shaper

- 4 min read
Jorin Vogel
Jorin Vogel

At Taleshape, we are building Shaper - an open source SQL-based tool to visualize data and build dashboards.

And we think Shaper is a great fit for building custom data analytics AI agents.

Let’s look at an agent in action:

  • Talk to the agent directly in Slack, and it uses Shaper to answer your data questions.
  • The agent can create anything from charts and Excel files to complete PDF reports, and then shares these files in Slack.
  • Since any chart or report in Shaper is just a SQL query, we can easily verify and reproduce any answer the agent gives.
Play

How It Works

Section titled “How It Works”
  1. The agent itself is a simple Python application that listens to Slack messages.
  2. When it receives a message, it uses an LLM to answer the user request.
  3. We give the LLM tools to use Shaper’s API to generate png/pdf/xlsx files directly from SQL queries.
  4. We then upload the generated files to Slack. While the LLM generates the SQL, it doesn’t touch the results, avoiding the risk of hallucinated data or charts.

Architecture Diagram

Since we control the agent and Shaper runs in our own infrastructure, we are also in full control of what data the LLM sees and which LLM we use. This setup can be easily adapted to fit the privacy and security needs of any organization.

Agents and Dashboards

Section titled “Agents and Dashboards”

Shaper’s interactive dashboards are great for tracking known metrics or giving customers an overview of their data.

But what do we do if we have ad-hoc questions or want to explore our data beyond what any existing dashboard was built for?

That’s where AI agents can help:

  • Build an agent that answers your business team’s data questions without the help of a data analyst.
  • Build an agent to explore new use cases for your data.

Shaper’s SQL-based approach means that instead of answering one-off questions your data team can focus on:

  1. Reviewing and verifying answers the agent gives.
  2. Turn any ad-hoc request into customer-facing dashboards and reusable reports.

Build Your Own Agent

Section titled “Build Your Own Agent”

Building your own agent not only means that you stay in full control of your data, but also that you can build an agent that is truly useful in practice. You can narrow the problem scope and build an agent for exactly the workflows most useful to you, integrating it into your processes with the context it needs.

Shaper provides the foundation to access data, create charts and reports, while ensuring correctness and keeping everything reproducible.

At Taleshape, we help our customers build systems exactly like this agent.

If you’d like to learn more, don’t hesitate to reach out.

Self-hosted Data Lake Platform with DuckDB, DuckLake and Shaper

- 6 min read
Jorin Vogel
Jorin Vogel

Do you need to store and analyze large amounts of data without breaking the bank? Do you need to run in your own infrastructure instead of relying on the big cloud providers?

You can build a simple data lake platform that scales from a $5 VM to petabytes of data with these three open source tools: DuckDB, DuckLake, and Shaper.

Architecture Diagram

What Does a Data Platform Need?

Section titled “What Does a Data Platform Need?”

A minimal data platform requires at least three components:

  1. Data Storage
  2. Data Ingestion and Transformation
  3. Data Analysis and Visualization

Let’s go over them one by one:

Data Storage

Section titled “Data Storage”

When you have only a small amount of data you can use DuckDB to directly query your production database such as Postgres, but as your data grows that gets slow and expensive.

One alternative is to replicate data to a data warehouse such as Snowflake, BigQuery, Redshift or even Clickhouse. But warehouses are complex to operate and expensive.

That’s where data lakes come in:

What’s a Data Lake?

Section titled “What’s a Data Lake?”

Building a data lake architecture means storing data as files in a cheap object storage such as S3.

Data is then stored in a format optimized for analytics such as Parquet files. And DuckDB can query those files really efficiently - fast enough for most use cases even with large amounts of data.

But Parquet files are read-only. To update data you need to create a new file. So you quickly end up with a large number of files that you need to manage and the need to rewrite files for better performance.

That’s why a data lake needs a metadata layer - a catalog of all the data.

Once you have a catalog, using a data lake feels just like using a database:

You create, update and insert into tables without thinking about Parquet files. The catalog automatically rewrites and optimizes the files for you as needed.

Why DuckLake?

Section titled “Why DuckLake?”

The most popular implementation of such a catalog is Apache Iceberg. Cloud providers offer Iceberg as a service (for example AWS S3 Tables and Cloudflare R2 Data Catalog).

And DuckDB has great support for Iceberg.

If your infrastructure provider already supports Iceberg on top of its object storage and you don’t mind paying the premium for that, that’s a great option for a data lake.

However, to run your own data lake on top of any object storage, DuckLake is much simpler than running your own Iceberg implementation.

DuckLake doesn’t require any additional services to run. It works with any object storage and supports many databases for its catalog.

If you have an S3-compatible service and a Postgres database, you can start using DuckLake without any new infrastructure.

Running DuckLake and Shaper

Section titled “Running DuckLake and Shaper”

We can run a complete data lake platform in a simple docker-compose setup:

  • Minio for object storage
  • Postgres for the DuckLake catalog
  • Shaper to run DuckDB and provide a UI for querying and scheduling jobs
docker-compose.yml
services:
  minio:
    image: minio/minio
    restart: unless-stopped
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    volumes:
      - minio_data:/data
    healthcheck:
      test: ["CMD", "mc", "ready", "local"]
      interval: 5s
      timeout: 5s
      retries: 10


  minio-init:
    image: minio/mc
    depends_on:
      minio:
        condition: service_healthy
    entrypoint: >
      /bin/sh -c "
        mc alias set local http://minio:9000 minioadmin minioadmin &&
        mc mb --ignore-existing local/ducklake &&
        echo 'Bucket ducklake ready.'
      "
    restart: "no"


  postgres:
    image: postgres:16
    restart: unless-stopped
    environment:
      POSTGRES_USER: ducklake
      POSTGRES_PASSWORD: ducklake
      POSTGRES_DB: ducklake
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ducklake -d ducklake"]
      interval: 5s
      timeout: 5s
      retries: 10


  shaper:
    image: taleshape/shaper
    restart: unless-stopped
    depends_on:
      postgres:
        condition: service_healthy
      minio-init:
        condition: service_completed_successfully
    ports:
      - "5454:5454" # UI → http://localhost:5454
    volumes:
      - ./init.sql:/init.sql:ro
    environment:
      SHAPER_INIT_SQL_FILE: /init.sql
      MINIO_KEY_ID: minioadmin
      MINIO_SECRET: minioadmin
      MINIO_ENDPOINT: minio:9000
      MINIO_BUCKET: ducklake
      PG_HOST: postgres
      PG_PORT: "5432"
      PG_DB: ducklake
      PG_USER: ducklake
      PG_PASSWORD: ducklake


volumes:
  minio_data:
  postgres_data:
  shaper_data:

We also need to configure Shaper to set up the secrets for DuckLake. We can do that in a separate init.sql file that Shaper executes on startup:

init.sql
CREATE OR REPLACE SECRET minio_secret (
    TYPE s3,
    KEY_ID '${MINIO_KEY_ID}',
    SECRET '${MINIO_SECRET}',
    REGION 'us-east-1',
    ENDPOINT '${MINIO_ENDPOINT}',
    URL_STYLE 'path',
    USE_SSL false
);


CREATE OR REPLACE SECRET pg_secret (
    TYPE postgres,
    HOST '${PG_HOST}',
    PORT ${PG_PORT},
    DATABASE '${PG_DB}',
    USER '${PG_USER}',
    PASSWORD '${PG_PASSWORD}'
);


CREATE SECRET ducklake_secret (
    TYPE ducklake,
    METADATA_PATH '',
    DATA_PATH 's3://${MINIO_BUCKET}/',
    METADATA_PARAMETERS MAP {'TYPE': 'postgres', 'SECRET': 'pg_secret'}
);


ATTACH 'ducklake:ducklake_secret' AS ducklake;

You can start the setup by running docker compose up -d in the directory where you created the docker-compose.yml and init.sql files.

Ingesting and Transforming Data

Section titled “Ingesting and Transforming Data”

We can use Shaper’s Task feature as a simple scheduling system to run SQL queries that ingest and transform data in our data lake:

  1. Open http://localhost:5454 in your browser.
  2. Click on New and select Task at the top.
  3. Copy and paste the following SQL:
SELECT NULL::SCHEDULE;
CREATE TABLE IF NOT EXISTS ducklake.sessions AS
    SELECT * FROM 'https://taleshape.com/sample-data/sessions.parquet';
  1. Click on “Create” at the top right and give the task a name such as “Ingest Sessions Data”.
  2. Now click on “Run” in the top right corner.

Congrats, you’ve loaded your first data into DuckLake!

For real use cases you would want to schedule tasks to run automatically and then you would use for example MERGE INTO to update your data lake continuously.

And of course not only Shaper can access DuckLake. For more complex setups you can introduce other tools for your data pipelines such as DBT and Dagster.

Analyze and Visualize Data

Section titled “Analyze and Visualize Data”

Once we have data in our data lake we can now query it.

Let’s build a first Shaper dashboard on top of DuckLake:

  1. In the Shaper UI click on New and select Dashboard at the top.
  2. Copy and paste the following SQL:
SELECT count(*) AS "Total Sessions"
FROM ducklake.sessions;


SELECT 'Sessions per Week'::LABEL;
SELECT
  date_trunc('week', created_at)::XAXIS,
  category::CATEGORY,
  count(*)::BARCHART_STACKED,
FROM ducklake.sessions GROUP BY ALL ORDER BY ALL;
  1. Click on “Create” at the top right and give the dashboard a name such as “Sessions Dashboard”.

Congrats, you just built your first dashboard on top of your data lake!

You can learn more about building dashboards in Shaper in the Getting Started Guide.

Deploying to Production

Section titled “Deploying to Production”

You now have a minimal, but complete data lake platform that you can build interesting data products on top of.

But for now this is only running on your computer locally. Now it’s time to deploy this setup into a production infrastructure.

The cheapest way to get started is to simply run all services on a single server, almost identical to how we did above.

But you wouldn’t need a data lake if you have so little data that it fits on a single machine.

Instead, you want to make use of an existing object storage service which almost every hosting provider offers.

Section titled “Recommended: Hetzner”

If you are looking for a more affordable hosting provider or prefer a European provider over the big American cloud providers, I recommend you look into Hetzner (referral to get free 20€).

You get a VM for 3.49€ and 1TB of object storage for 4.99€, hosted in Germany.

Just be careful that there are some gotchas with Hetzner’s object storage. To make sure your setup is properly secured, check out this detailed guide by my friend Floyd:

DuckLake on Hetzner

And if you are looking for fully-managed Shaper connected to your data lake and integrated into your systems, check out Taleshape’s Managed Hosting and Support Plans.

Analytics Dashboards as Code

- 2 min read
Jorin Vogel
Jorin Vogel

Shaper has always been about building dashboards using SQL.

And it’s just natural to manage SQL as files just like any other code.

With the latest release of Shaper you can now deploy dashboards from files,
and live-preview dashboard changes.

You get to:

  • use your favorite editor - including its AI features
  • track dashboards in Git
  • collaborate with your team using pull requests
  • deploy dashboards via CI/CD

The workflow in action:

Play

Getting Started

Section titled “Getting Started”

Getting started and sharing Shaper projects is now simpler than ever since everything is just files. Let’s give it a try:

  1. Clone this demo Git repository:
    Terminal window
    git@github.com:taleshape-com/demo-project.git
  2. Install Shaper in the project directory:
    Terminal window
    make install
  3. Start a local Shaper server to serve the dashboards:
    Terminal window
    make serve
  4. Deploy the two dashboards in the dashboards/ folder by running the deploy command in a second terminal:
    Terminal window
    make deploy
  5. Live-preview changes by running the dev file watcher in the second terminal:
    Terminal window
    make dev

    Now edit or create any SQL file in the dashboards/ folder and see the changes live in your browser.

Learn More

Section titled “Learn More”

Usually you will run Shaper on a server. Then you only need shaper dev locally when developing dashboards. Once you configured authentication for your Shaper instance, the dev command will automatically prompt you to login and authenticate. And instead of running deploy manually, you can use the Shaper Github Action to deploy dashboards automatically in your CI/CD pipeline.

All details are in the documentation.

And as always - we are happy to hear about any feedback via socials or Github issues or discussions!