Sandcastle

Note

This repository is archived. Sandcastle was a prototype that grew into Vermilion, which is now available on the App Store. Vermilion is not open source. You can view or open issues on Vermilion GitHub repo if you would like to do so.

Sandcastle is a prototype iOS app built on top of the Gemini Live API.

Tools

In particular, the app aims to demonstrate Tool use.

To demonstrate Tool use, the app includes a UI called "playground" which can be interacted with via Tools with the following pseudo-code declarations:

/// Set if the playground should be shown to the user
function playground_set_is_showing(boolean should_show) -> ()

enum Color {
    black
    blue
    brown
    cyan
    gray
    green
    indigo
    mint
    orange
    pink
    purple
    red
    teal
    white
    yellow
}

/// Set the color used in the playground
function playground_set_color(enum Color color) -> ()

The user could use the prompt "Please set the playground to the color that trees are commonly depicted as", which likely would result in the Tool call: playground_set_color(Color.green)

Demo

demo.mp4