Models

Sonoe 1.1

How it decides

Each option is read on its own together with the state and the question, and scored. The scores of a question's options become probabilities with a softmax. Because options are scored independently, their order doesn't matter and a Choice can have hundreds of them.

Probabilities are calibrated: after training, each question type gets its own temperature, fitted on phrasings the model never saw, so a 0.9 means about 90%. Each type also gets an abstention threshold, used by /v1/decide.

What it's good at, and what it isn't

• Choice is its strongest type. In its first two products, Sonoe 1.1 is right about 90% of the time when it answers: 90.5% classifying searches into kinds of place, and 90.9% on Puerto Rico's health and emergency directory.
• Yes/no over a structured state improved a lot in 1.1 (from 48% to 94.7% on its test), by first finding the field the question is about.
• Scores and free-form yes/no questions are weaker. Confidence is often low and wording matters. Check them on your data.
• It's sensitive to wording. Plain words and described options work best. See Questions.
• It reads short texts. 256 tokens per option: a message, a search, a record. Not a contract.
• It's trained on synthetic data in Spanish and English (and their mix), with Puerto Rico Spanish, Portuguese, French, Haitian Creole and Chinese in some tasks. It handles other languages but hasn't been measured on them.

These results are measured on held-out phrasings of synthetic data. The real world is messier: measure on your own data before relying on Sonoe.

Aliases

An alias moves when a new model ships, so its answers can change without any change in your code. Each answer's model field says which version answered. If you've tuned thresholds on one version, send its id (sonoe-1.1.0) and move when you're ready.