Posts

Building scalable data pipelines with Kedro

data engineering

Kedro is a toolbox for production-ready data science. It is an open-source Python framework like Ibis, and together you can bring the portability and scale of Ibis to the…

Modern, hybrid, open analytics

As a Python data user, I’ve wanted a more modular, composable, and scalable ecosystem. I think it’s here. Wes McKinney released pandas c. 2009 to bring dataframes into…

Using one Python dataframe API to take the billion row challenge with DuckDB, Polars, and DataFusion

This is an implementation of the The One Billion Row Challenge:

Backend agnostic arrays

This is a redux of a previous post showing Ibis’s portability in action.

Geospatial analysis with Ibis and DuckDB (redux)

Spatial Dev Guru wrote a great tutorial that walks you through a step-by-step geospatial analysis of bike sharing data using DuckDB.

Naty Clementi and Gil Forsyth

Announcing Zulip for Ibis community chat

The Ibis project has moved to Zulip for its community chat! We’ve been testing it out for a few months and are happy with the results. From the Zulip repository’s README:

Ibis versus X: Performance across the ecosystem part 2

TL; DR: Ibis supports both Polars and DataFusion. Both backends are have about the same runtime performance, and lag far behind DuckDB on this workload. There’s negligible…

Ibis + DuckDB geospatial: a match made on Earth

Ibis now has support for DuckDB geospatial functions!

Ibis versus X: Performance across the ecosystem part 1

TL; DR: Ibis has a lot of great backends. They’re all good at different things. For working with local data, it’s hard to beat DuckDB on feature set and performance.

dbt-ibis: Write your dbt models using Ibis

data engineering

dbt has revolutionized how transformations are orchestrated and managed within modern data warehouses. Initially released in 2016, dbt quickly gained traction within the…

Querying every file in every release on the Python Package Index (redux)

Seth Larson wrote a great blog post on querying a PyPI dataset to look for trends in the use of memory-safe languages in Python.

Working with arrays in Google BigQuery

Ibis and BigQuery have worked well together for years.

Icy IO: loading local files with Snowflake

It can be challenging to load local files into Snowflake from Python.

Ibis v6.1.0

Ibis 6.1.0 is a minor release that includes new features, backend improvements, bug fixes, documentation improvements, and refactors. We are excited to see further adoption…

Ibis v6.0.0

Ibis 6.0.0 adds the Oracle backend, revamped UDF support, and many new features. This release also includes a number of refactors, bug fixes, and performance improvements.…

Ibis on 🔥: Supercharge Your Workflow with DuckDB and PyTorch

machine learning

In this blog post we show how to leverage ecosystem tools to build an end-to-end ML pipeline using Ibis, DuckDB and PyTorch.

Exploring campaign finance data

data engineering

Hi! My name is Nick Crews, and I’m a data engineer that looks at public campaign finance data.

Ibis sneak peek: writing to files

Ibis 5.0 is coming soon and will offer new functionality and fixes to users. To enhance clarity around this process, we’re sharing a sneak peek into what we’re working on.

Ibis sneak peek: examples

Ibis has been moving quickly to provide a powerful but easy-to-use interface for interacting with analytical engines. However, as we’re approaching the 5.0 release of Ibis…

Maximizing productivity with selectors

Before Ibis 5.0 it’s been challenging to concisely express whole-table operations with ibis. Happily this is no longer the case in ibis 5.0.

Ibis + Substrait + DuckDB

Ibis strives to provide a consistent interface for interacting with a multitude of different analytical execution engines, most of which (but not all) speak some dialect of…

Analysis of Ibis’s CI performance

continuous integration

data engineering

This notebook takes you through an analysis of Ibis’s CI data using ibis on top of Google BigQuery.

Ibis v4.0.0

Ibis 4.0 has officially been released as the latest version of the package. This release includes several new backends, improved functionality, and some major internal…

`ffill` and `bfill` using Ibis

window functions

Suppose you have a table of data mapping events and dates to values, and that this data contains gaps in values.

Ibis v3.1.0

Ibis 3.1 has officially been released as the latest version of the package. With this release comes new convenience features, increased backend operation coverage and a…

Marlene Mhangami

Ibis v3.0.0

The latest version of Ibis, version 3.0.0, has just been released! This post highlights some of the new features, breaking changes, and performance improvements that come…

Marlene Mhangami