We’re using Python, let’s use Python

We’re already using Python for Ibis and Marvin. Let’s use it for auxillary tools. We’ll setup our data and AI platform connections and some simple example data to work with.

1import ibis
import marvin

from dotenv import load_dotenv

2load_dotenv()

3con = ibis.connect("duckdb://penguins.ddb")
t = ibis.examples.penguins.fetch()
t = con.create_table("penguins", t.to_pyarrow(), overwrite=True)

1import ibis
import marvin

from ibis.expr.schema import Schema
from ibis.expr.types.relations import Table

2ibis.options.interactive = True
marvin.settings.llm_model = "openai/gpt-4"

3con = ibis.connect("duckdb://penguins.ddb")
t = con.table("penguins")

Filesystem tools

Internet tools

AI-powered tools

Introducing Ibis Birdbrain

Introduce the bot. Need to overview the tools here I think, but should probably skip most details.

A comparison with MLOps

TODO: point on how most of the work is not ML

Before “MLOps” was a standard term, the Sculley et al paper from 2015 described the key issues with building real-world ML systems.

You can substitute “LLM” for “ML” in the above figure.

Next steps

You can get involved with Ibis Birdbrain, our open-source data & AI project for building next-generation natural language interfaces to data.

Standards

When things don’t work as they should, it often means that standards are absent.

- The International Organization for Standardization (ISO)

- The Composable Codex

The standard that has emerged from the recent LLM applications is the power of a conversation in interacting with a computer. If we can give that interaction tht ability to interact with data platforms, we can achieve a lot through natural language.

Composable data (and AI) systems are MICE

Composable data systems are MICE: modular, interoperable, customizable, and extensible.

With any new tech, there’s a lot of duplication of effort and reinvention of wheels. Ibis Birdbrain aims to be an open-source standard for building data and AI systems in Python that adheres to MICE best practices and works with any data and AI platform. It achieves this by building on top of Ibis and Marvin, handling calls to the data and AI platforms respectively. Ibis and Marvin are themselves built on many other open-source projects, but provide the right level of abstraction for building composable data and AI systems, independently. Ibis Birdbrain glues them together enables a natural language interface that’s interoperable between 18+ data platforms and any AI platform Marvin supports.

glue that enables working via natural language with any data platform Ibis supports and any AI platform Marvin supports.

Looking ahead

I am not a software engineer, I just saw a cool application of Ibis and Marvin. The objective of releasing Ibis Birdbrain as an open-source project under Ibis governance is to enable the community to build a standard for composable data and AI systems. My hope is for the Ibis, Marvin, and broader data/AI Python communities to build the standard in the open, and properietary applications on top of those standards. This is important for a composable data ecosystem that can be modular, interoperable, customizable, and extensible.

How is Ibis Birdbrain MICE now? What’s next?

Next steps

You can get involved with Ibis Birdbrain, our open-source data & AI project for building next-generation natural language interfaces to data.

Read the next post in this series, building Ibis Birdbrain.

Context

Context is a fancy way of talking about the input to a LLM.

Calls

We make calls with inputs to functions or systems and get outputs. We can think of calling the LLM with our input (context) and getting an output (text).

Computations

A function or system often computes something. We can be pedantic about calls versus computations, but in general the connotation around computations is more time and resource intensive than a call. At the end of the day, they will both take some computer cycles.

Retrieval augmented generation (RAG)

Instead of you typing out context for the bot, we can retrieve context from somewhere, augment our strings sent to the bot with this context, and then generate a response from the bot.

As a contrived example, instead of saying “The capitol of foo is bar”, we can retrieve the capitol of foo from a database, augment it with our context, and then generate a response from the bot. You may notice that we already did this in the firt post in the series – let’s review that code again:

from ibis.expr.schema import Schema
from ibis.expr.types.relations import Table


@marvin.ai_fn
def sql_select(
    text: str, table_name: str = t.get_name(), schema: Schema = t.schema()
) -> str:
    """writes the SQL SELECT statement to query the table according to the text"""


query = "the unique combination of species and islands"
sql = sql_select(query).strip(";")
sql

'SELECT DISTINCT species, island FROM penguins'

Notice that we retrieved the table name and schema with calls to the Ibis table (t.get_name() and t.schema()). We then augment our context (the query in natural language) with this information and generate a response from the bot.

This works reasonably well for simple SQL queries:

t.sql(sql)

┏━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ species   ┃ island    ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━┩
│ string    │ string    │
├───────────┼───────────┤
│ Adelie    │ Torgersen │
│ Adelie    │ Biscoe    │
│ Adelie    │ Dream     │
│ Gentoo    │ Biscoe    │
│ Chinstrap │ Dream     │
└───────────┴───────────┘

I would argue in this case there wasn’t any real computation done by our calls to the Ibis table – we were just retrieving some relatively static metadata – but we could have done some more complex computations (on any of 18+ data platforms).

Thought leadership

TODO: human rewrite

In the realm of Generative AI, particularly when working with Language Learning Models (LLMs), understanding the concept of ‘context’ is crucial. Context, in this domain, refers to the inputs that are fed into an LLM, and the corresponding outputs they generate. This post breaks down the complexities of this process into understandable fragments, including retrieval of context, its augmentation, and, thereafter, the generation of a response.

An illustrative example is provided, showcasing a database interaction. It demonstrates how the data retrieved can be used to augment the context before the bot generates a response. This valuable insight underlines the practical application of the theory, reinforcing the understanding of the readers.

We also venture into the difference between simple static metadata retrieval and the more intricate computations. This distinction echoes the breadth and depth of the processes involved in Generative AI.

As we continue to explore and unravel the potential of Generative AI and LLMs, this post serves as a fundamental building block. It creates a pathway for enthusiasts and professionals alike to delve deeper into this exciting field. By breaking down complex concepts into comprehensible segments, it fosters an environment of learning and growth.

This marks just the beginning of our journey into the world of Generative AI. As we dig deeper, we will continue to explore, learn and share with our readers. Stay tuned for more insightful content in this series. [1]

[1] https://github.com/ibis-project/ibis-birdbrain

Next steps

You can get involved with Ibis Birdbrain, our open-source data & AI project for building next-generation natural language interfaces to data.

Read the next post in this series.

Approaches

When discussed at Voltron Data, we identified three distinct approaches to applying LLMs to data analytics that can be implemented today:

1. LLM writes an analytic code
2. LLM writes an analytic subroutine
3. Use LLM in an analytic subroutine

While these three approaches are not an exhaustive list of how LLMs can be applied to data, they can be easily understood and implemented with Ibis and Marvin in a few lines of code. Together with these two open-source tools, we can build a natural language interface for data analytics that supports 18+ backends.

But first, let’s demonstrate the three approaches.

1import ibis
import marvin

from rich import print
from time import sleep
from dotenv import load_dotenv

2load_dotenv()

3con = ibis.connect("duckdb://penguins.ddb")
t = ibis.examples.penguins.fetch()
t = con.create_table("penguins", t.to_pyarrow(), overwrite=True)

1import ibis
import marvin

from ibis.expr.schema import Schema
from ibis.expr.types.relations import Table


2ibis.options.interactive = True
marvin.settings.llm_model = "openai/gpt-4"

1@marvin.ai_fn
def _generate_sql_select(
    text: str, table_name: str, table_schema: Schema
) -> str:
    """Generate SQL SELECT from text."""


2def sql_from_text(text: str, t: Table) -> Table:
    """Run SQL from text."""
    return t.sql(_generate_sql_select(text, t.get_name(), t.schema()).strip(";"))

t2 = sql_from_text("the unique combination of species and islands", t)
t2

┏━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ species   ┃ island    ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━┩
│ string    │ string    │
├───────────┼───────────┤
│ Adelie    │ Torgersen │
│ Adelie    │ Biscoe    │
│ Adelie    │ Dream     │
│ Gentoo    │ Biscoe    │
│ Chinstrap │ Dream     │
└───────────┴───────────┘

1sleep(3)

t3 = sql_from_text(
    "the unique combination of species and islands, with their counts, ordered from highest to lowest, and name that column just 'count'",
    t,
)
t3

┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┓
┃ species   ┃ island    ┃ count ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━┩
│ string    │ string    │ int64 │
├───────────┼───────────┼───────┤
│ Gentoo    │ Biscoe    │   124 │
│ Chinstrap │ Dream     │    68 │
│ Adelie    │ Dream     │    56 │
│ Adelie    │ Torgersen │    52 │
│ Adelie    │ Biscoe    │    44 │
└───────────┴───────────┴───────┘

1sleep(3)

1@marvin.ai_fn
def _generate_python_function(text: str) -> str:
    """Generate a simple, typed, correct Python function from text."""


2def create_udf_from_text(text: str) -> str:
    """Create a UDF from text."""
    return f"""
import ibis

@ibis.udf.scalar.python
{_generate_python_function(text)}
""".strip()

udf = create_udf_from_text(
    "a function named count_vowels that given an input string, returns an int w/ the number of vowels (y_included as a boolean option defaulted to False)"
)
print(udf)
exec(udf)

import ibis

@ibis.udf.scalar.python
def count_vowels(input_string: str, y_included: bool = False) -> int:
    """Given an input string, it returns the number of vowels. y can be included as a vowel."""
    vowels = 'aeiou'
    if y_included:
        vowels += 'y'
    return sum(1 for char in input_string if char in vowels)

1sleep(3)

t4 = t3.mutate(
    species_vowel_count=count_vowels(t3.species),
    island_vowel_count=count_vowels(t3.island),
)
t4

┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓
┃ species   ┃ island    ┃ count ┃ species_vowel_count ┃ island_vowel_count ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩
│ string    │ string    │ int64 │ int64               │ int64              │
├───────────┼───────────┼───────┼─────────────────────┼────────────────────┤
│ Gentoo    │ Biscoe    │   124 │                   3 │                  3 │
│ Chinstrap │ Dream     │    68 │                   2 │                  2 │
│ Adelie    │ Dream     │    56 │                   3 │                  2 │
│ Adelie    │ Torgersen │    52 │                   3 │                  3 │
│ Adelie    │ Biscoe    │    44 │                   3 │                  3 │
└───────────┴───────────┴───────┴─────────────────────┴────────────────────┘

1sleep(3)

1from pydantic import BaseModel, Field

# decrease cost
2marvin.settings.llm_model = "openai/gpt-3.5-turbo-16k"


3@marvin.ai_model
class VowelCounter(BaseModel):
    """Count vowels in a string."""

    include_y: bool = Field(False, description="Include 'y' as a vowel.")
    # num_a: int = Field(..., description="The number of 'a' vowels.")
    # num_e: int = Field(..., description="The number of 'e' vowels.")
    # num_i: int = Field(..., description="The number of 'i' vowels.")
    # num_o: int = Field(..., description="The number of 'o' vowels.")
    # num_u: int = Field(..., description="The number of 'u' vowels.")
    # num_y: int = Field(..., description="The number of 'y' vowels.")
    num_total: int = Field(..., description="The total number of vowels.")


4VowelCounter("hello world")

VowelCounter(include_y=False, num_total=3)

udf = create_udf_from_text(
    "a function named count_vowels_ai that given an input string, calls VowelCounter on it and returns the num_total attribute of that result"
)
print(udf)
exec(udf)

import ibis

@ibis.udf.scalar.python
def count_vowels_ai(input_string: str) -> int:
    result = VowelCounter(input_string)
    return result.num_total

1sleep(3)

t5 = t3.mutate(
    species_vowel_count=count_vowels_ai(t3.species),
    island_vowel_count=count_vowels_ai(t3.island),
)
t5

┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓
┃ species   ┃ island    ┃ count ┃ species_vowel_count ┃ island_vowel_count ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩
│ string    │ string    │ int64 │ int64               │ int64              │
├───────────┼───────────┼───────┼─────────────────────┼────────────────────┤
│ Gentoo    │ Biscoe    │   124 │                   3 │                  3 │
│ Chinstrap │ Dream     │    68 │                   2 │                  1 │
│ Adelie    │ Dream     │    56 │                   3 │                  2 │
│ Adelie    │ Torgersen │    52 │                   3 │                  3 │
│ Adelie    │ Biscoe    │    44 │                   3 │                  3 │
└───────────┴───────────┴───────┴─────────────────────┴────────────────────┘

1sleep(3)

Summary

To summarize this post:

from rich import print

with open("index.qmd", "r") as f:
    self_text = f.read()

# increease accuracy
marvin.settings.llm_model = "openai/gpt-4"

@marvin.ai_model
class Summary(BaseModel):
    """Summary of text."""

    summary_line: str = Field(..., description="The one-line summary of the text.")
    summary_paragraph: str = Field(
        ..., description="The one-paragraph summary of the text."
    )
    conclusion: str = Field(
        ..., description="The conclusion the reader should draw from the text."
    )
    key_points: list[str] = Field(..., description="The key points of the text.")
    critiques: list[str] = Field(
        ..., description="Professional, fair critiques of the text."
    )
    suggested_improvements: list[str] = Field(
        ..., description="Suggested improvements for the text."
    )
    sentiment: float = Field(..., description="The sentiment of the text.")
    sentiment_label: str = Field(..., description="The sentiment label of the text.")
    author_bias: str = Field(..., description="The author bias of the text.")


print(Summary(self_text))

Summary(
    summary_line="The blog post titled 'Three approaches' by Cody Peterson, dated 2023-10-13, discusses three 
distinct approaches to applying Language Learning Models (LLMs) to data analytics using open-source tools Ibis and 
Marvin.",
    summary_paragraph='The author explains three approaches to applying LLMs to data analytics. The first approach 
is letting the LLM write an analytic code, specifically generating SQL. The second approach involves letting LLM 
write an analytic subroutine, demonstrated by writing Python code for user-defined functions. The third approach 
uses LLM in an analytic subroutine, illustrated by calling the LLM once-per-row in a table via a subroutine. Each 
method has potential applications and limitations, and the choice of approach will depend on the specific 
requirements of the data analytics task.',
    conclusion='LLMs can be used to transform and analyze data in various ways. By using these approaches in 
combination with open-source tools like Ibis and Marvin, it is possible to build a natural language interface for 
data analytics that supports multiple backends. The author concludes by introducing an open-source data & AI 
project for building next-generation natural language interfaces to data, Ibis Birdbrain.',
    key_points=[
        'The LLM can write an analytic code, particularly generating SQL.',
        'The LLM can also write an analytic subroutine, exemplified by writing Python code for user-defined 
functions.',
        'The LLM can be used in an analytic subroutine, shown by calling the LLM once-per-row in a table via a 
subroutine.',
        'The author suggests that each approach has its potential applications and limitations.',
        'The choice of approach will depend on the specific requirements of the data analytics task.'
    ],
    critiques=[
        'The author could have provided more concrete examples or case studies to illustrate the application of 
these approaches.',
        'A comparison of the advantages and disadvantages of each approach would have been helpful.'
    ],
    suggested_improvements=[
        'The author could include real-world applications or examples of each approach.',
        'A detailed comparison of the pros and cons of each approach would provide better guidance for readers.'
    ],
    sentiment=0.25,
    sentiment_label='Neutral',
    author_bias='The author shows a positive bias towards the use of open-source tools, Ibis and Marvin, for data 
analytics.'
)

Next steps

You can get involved with Ibis Birdbrain, our open-source data & AI project for building next-generation natural language interfaces to data.

Read the next post in this series.

Marvin

Marvin is an AI engineering framework that makes it easy to build up to an interactive conversational application.

Marvin makes calls to an AI platform. You typically use an API key set as an environment variable – in this case, we’ll load a .env file that contians secrets for the AI platform that Marvin will use. We also set the large language model model.

1import marvin

from rich import print
from time import sleep
from dotenv import load_dotenv

2load_dotenv()

# increase accuracy
3marvin.settings.llm_model = "openai/gpt-4"
# decrease cost
# marvin.settings.llm_model = "openai/gpt-3.5-turbo"

4test_str = "working with data and LLMs on 18+ data platforms is easy!"
test_str

'working with data and LLMs on 18+ data platforms is easy!'

@marvin.ai_fn
def translate(text: str, from_: str = "English", to: str = "Spanish") -> str:
    """translates the text"""

translate(test_str)

'trabajar con datos y LLMs en más de 18 plataformas de datos es fácil!'

1sleep(1)

translate(translate(test_str), from_="Spanish", to="English")

'Working with data and LLMs on more than 18 data platforms is easy!'

1sleep(3)

from pydantic import BaseModel, Field

# decrease cost
marvin.settings.llm_model = "openai/gpt-3.5-turbo"

@marvin.ai_model
class ExtractParts(BaseModel):
    """Extracts parts of a sentence"""
    subject: str = Field(..., description="The subject of the sentence.")
    objects: list[str] = Field(..., description="The objects of the sentence.")
    predicate: str = Field(..., description="The predicate of the sentence.")
    modifiers: list[str] = Field(..., description="The modifiers of the sentence.")

ExtractParts(test_str)

ExtractParts(subject='working', objects=['data', 'LLMs'], predicate='is', modifiers=['on 18+ data platforms', 'easy'])

1sleep(1)

from enum import Enum

# increase accuracy
marvin.settings.llm_model = "openai/gpt-4"

@marvin.ai_classifier
class IdentifyLanguage(Enum):
    """Identifies the language of the text"""

    english = "English"
    spanish = "Spanish"


IdentifyLanguage(test_str).value

'English'

1sleep(1)

IdentifyLanguage(translate(test_str)).value

'Spanish'

1sleep(3)

Ibis

Ibis is the portable Python dataframe library that enables Ibis Birdbrain to work on many data platforms at native scale.

Ibis makes calls to a data platform, providing an API but pushing the compute to (local or remote) query engines and storage. DuckDB is the default and we’ll typically use it for demo puroses. You can work with an in-memory instance, but we’ll often create a database file from example data:

1import ibis

2con = ibis.connect("duckdb://penguins.ddb")
t = ibis.examples.penguins.fetch()
t = con.create_table("penguins", t.to_pyarrow(), overwrite=True)

You will typically connect to an existing data platform via your corresponding Ibis backend and have access to a number of tables:

1import ibis

2ibis.options.interactive = True

3con = ibis.connect("duckdb://penguins.ddb")
t = con.table("penguins")

con

<ibis.backends.duckdb.Backend at 0x16a17af10>

con.list_tables()

['penguins']

backends = [entrypoint.name for entrypoint in ibis.util.backend_entry_points()]
backends

['bigquery',
 'clickhouse',
 'dask',
 'datafusion',
 'druid',
 'duckdb',
 'flink',
 'impala',
 'mssql',
 'mysql',
 'oracle',
 'pandas',
 'polars',
 'postgres',
 'pyspark',
 'snowflake',
 'sqlite',
 'trino']

t

┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓
┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃
┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩
│ string  │ string    │ float64        │ float64       │ int64             │ int64       │ string │ int64 │
├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤
│ Adelie  │ Torgersen │           39.1 │          18.7 │               181 │        3750 │ male   │  2007 │
│ Adelie  │ Torgersen │           39.5 │          17.4 │               186 │        3800 │ female │  2007 │
│ Adelie  │ Torgersen │           40.3 │          18.0 │               195 │        3250 │ female │  2007 │
│ Adelie  │ Torgersen │            nan │           nan │              NULL │        NULL │ NULL   │  2007 │
│ Adelie  │ Torgersen │           36.7 │          19.3 │               193 │        3450 │ female │  2007 │
│ Adelie  │ Torgersen │           39.3 │          20.6 │               190 │        3650 │ male   │  2007 │
│ Adelie  │ Torgersen │           38.9 │          17.8 │               181 │        3625 │ female │  2007 │
│ Adelie  │ Torgersen │           39.2 │          19.6 │               195 │        4675 │ male   │  2007 │
│ Adelie  │ Torgersen │           34.1 │          18.1 │               193 │        3475 │ NULL   │  2007 │
│ Adelie  │ Torgersen │           42.0 │          20.2 │               190 │        4250 │ NULL   │  2007 │
│ …       │ …         │              … │             … │                 … │           … │ …      │     … │
└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘

t.schema()

ibis.Schema {
  species            string
  island             string
  bill_length_mm     float64
  bill_depth_mm      float64
  flipper_length_mm  int64
  body_mass_g        int64
  sex                string
  year               int64
}

LLMs and data: Marvin and Ibis

You can use Marvin and Ibis together to easily apply LLMs to data.

from ibis.expr.schema import Schema
from ibis.expr.types.relations import Table

@marvin.ai_fn
def sql_select(
    text: str, table_name: str = t.get_name(), schema: Schema = t.schema()
) -> str:
    """writes the SQL SELECT statement to query the table according to the text"""


query = "the unique combination of species and islands"
sql = sql_select(query).strip(";")
sql

'SELECT DISTINCT species, island FROM penguins'

t.sql(sql)

┏━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ species   ┃ island    ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━┩
│ string    │ string    │
├───────────┼───────────┤
│ Adelie    │ Torgersen │
│ Adelie    │ Biscoe    │
│ Adelie    │ Dream     │
│ Gentoo    │ Biscoe    │
│ Chinstrap │ Dream     │
└───────────┴───────────┘

1sleep(3)

t.sql(sql_select(query + " and include their counts in from highest to lowest").strip(";"))

┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┓
┃ species   ┃ island    ┃ count ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━┩
│ string    │ string    │ int64 │
├───────────┼───────────┼───────┤
│ Gentoo    │ Biscoe    │   124 │
│ Chinstrap │ Dream     │    68 │
│ Adelie    │ Dream     │    56 │
│ Adelie    │ Torgersen │    52 │
│ Adelie    │ Biscoe    │    44 │
└───────────┴───────────┴───────┘

Next steps

You can get involved with Ibis Birdbrain, our open-source data & AI project for building next-generation natural language interfaces to data.

Read the next post in this series.