Mirascope
Product design, 2022 to 2025
Mirascope, which started as SOTAI, brought us back for three projects over two and a half years, from a no-code machine learning platform to an open-source tool for LLM developers.
Traces
5 calls, newest firstanswer_question v2
$0.000061,410msInput
What were our sales last quarter?
Output
I don't have access to that information, but most businesses see seasonal changes.
- 2.5 years
- coming back
- 3
- projects
- Apr 2023
- Artifice on Show HN
Where they were
The founders came out of Google AI and built tools for people who aren't data scientists. Their first product, Artifice, let anyone build interpretable machine learning models without code.
When we joined, Artifice was live and a second product was a prototype. Both worked, but they were hard to follow for anyone outside machine learning. We redesigned both so that people outside machine learning could see what each one was for and start using it.
The decision
In Lilypad, a team judges each answer from an LLM right beside it. A reviewer marks each answer pass or fail against the team's own criteria and gives a reason. That shows the team whether a new version of a prompt does better than the last one.
answer_question
Input
question
What is the capital of France?
tone
witty
Output
France has a long and rich history, and its capital has moved over the centuries, but these days most people would agree that it is Paris.
What shipped
First came Artifice, which launched on Hacker News in April 2023. Then the design library behind their products, which we took through two versions. When the team moved to LLM tools, we designed Lilypad, an open-source tool for versioning, tracing and annotating LLM calls, which sits alongside Mirascope.
How we worked together
They brought us in project by project, whenever they had something new to shape.
“Kelvin is an exceptional designer who worked closely with us on various projects for SOTAI… he went above and beyond to ensure that we were completely satisfied with the final product.”