---
license: apache-2.0
library_name: transformers
pipeline_tag: text-classification
base_model: Qwen/Qwen3.5-0.8B
base_model_relation: finetune
datasets:
  - lothnic/themis-decision-mixture
tags:
  - themis
  - decision-model
  - structured-prediction
  - probabilities
  - text-only
  - custom_code
  - safetensors
  - lora
---

# Themis 0.8B

**A small, text-only model for typed decisions.** Give Themis a case, questions, and their possible answers. It returns a probability distribution for each question in one forward pass. It supports named choices, yes/no decisions, and ordered scores, without generating or parsing answer text.

This repository contains the **complete model weights**, tokenizer, decision encoder, calibration settings, and inference code. No separate base-model download is required. The trained LoRA layers are retained inside the complete checkpoint to preserve the numerical path used for evaluation; this is a self-contained model release, not an adapter-only repository.

| Property | Value |
|---|---|
| Developer | [lothnic](https://huggingface.co/lothnic) |
| Version | Themis v0 / 0.8B |
| Backbone | Qwen3.5-0.8B text decoder |
| Active parameters | 752,393,024 base + 21,645,312 adapter parameters |
| Inputs | A text state and a mapping of typed questions |
| Outputs | Answers, named probabilities, confidence; expected value for scores |
| Rendered context | 2,048 tokens, including questions, options, and template |
| Options per question | 2–255; constrained by the context limit |
| Modalities | Text only |
| Released checkpoint | Step 4,153, selected by validation soft cross-entropy |
| License | Apache 2.0; see [LICENSE](LICENSE) and [NOTICE](NOTICE) |

## Quick start

Use Python 3.12. The following versions are the tested inference environment:

```bash
pip install torch==2.14.1 transformers==5.19.0 peft==0.21.2 tokenizers==0.23.3 safetensors huggingface_hub
# Optional CUDA acceleration used in the reported local evaluations:
pip install flash-linear-attention==0.5.2
```

```python
import torch
from transformers import AutoModel, AutoTokenizer

repo = "lothnic/themis-0.8b"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(
    repo,
    trust_remote_code=True,
    dtype=torch.bfloat16,
).to("cuda").eval()

result = model.predict(
    state="A customer was charged twice for one order and asks for a refund.",
    questions={
        "route": {
            "type": "choice",
            "instructions": "Which team should handle this request?",
            "criteria": {
                "billing": "Charges, payments, and refunds",
                "technical": "Software faults and account access",
                "sales": "Product information and new purchases",
            },
        },
        "needs_review": {
            "type": "noul",
            "instructions": "Does the reported duplicate charge need investigation?",
            "criteria": {"true": "Investigation needed", "false": "No investigation needed"},
        },
        "urgency": {
            "type": "score",
            "instructions": "Rate urgency using the following ordered levels.",
            "criteria": ["Low", "Medium", "High"],
        },
    },
    tokenizer=tokenizer,
)
print(result)
```

`trust_remote_code=True` loads the decision architecture and encoder shipped here. Review the source and pin `revision` to a release commit for reproducible use. This model uses a custom typed-request interface: the standard string-label `pipeline("text-classification")` interface and `.generate()` are not supported.

For CPU inference, load with `dtype=torch.float32` and leave the model on CPU. CPU results can differ slightly from BF16 CUDA results. CUDA BF16 on an RTX 4060 Laptop GPU is the locally verified reference environment. Other accelerators and quantized runtimes have not been validated.

### Response contract

The response has `model`, `answers`, `calibrated`, and `input_tokens` fields. `answers` is keyed by the original question names. Each answer contains `type`, `answer`, `probabilities`, and `confidence`.

- **`choice`**: `criteria` is a mapping from caller-defined option names to descriptions. `answer` is an original option name, and probabilities use those names.
- **`noul`**: yes/no. `answer` is a Boolean. Probability keys are always `"true"`, then `"false"`; the internal distribution is `[P(true), P(false)]`.
- **`score`**: `criteria` is a list ordered from lowest to highest. `answer` is the most probable **zero-based level index**, probability keys are string indices, and `expected_score` is the probability-weighted mean index. Convert this index to your own numeric scale explicitly.

Calibration is enabled by default. Pass `calibrated=False` to `predict()` for raw probabilities. Positive temperature scaling preserves each question's most probable answer, but can change an expected score or a threshold-based decision.

Options are supplied at inference time; there is no single global label vocabulary. Requests over 2,048 rendered tokens raise an error instead of silently truncating the evidence. Literal special-token strings in user-supplied text are escaped by the encoder.

## How it works

Themis uses the 24-layer Qwen3.5 text decoder: 18 gated linear-attention layers and six full-attention layers. Full-attention layers receive bidirectional attention masks over real tokens; linear-attention layers retain their forward recurrence. Padding is excluded from the full-attention keys.

The encoder renders the case, every question, and all answer descriptions, followed by fixed `<answer>` placeholders. For each question, the hidden state immediately before its placeholder is scored against the frozen embedding rows for that question's valid answer codes. Softmax is computed over those options only. The tokenizer mapping verifies 255 distinct single-token codes.

The `<answer>` embedding was initialized from the existing embedding mean and frozen. The base embeddings and output-code vectors were frozen during training; LoRA updated text-decoder projections. The released model uses the same encoding, masks, and readout as training. It computes no vocabulary-wide language-model logits and contains no vision tower or auxiliary multi-token prediction weights.

Multiple questions share a rendered request, so their surrounding context can affect one another. The response probabilities describe the provided alternatives; they are not a guarantee that those alternatives include the right action.

## Training and checkpoint selection

Base: [`Qwen/Qwen3.5-0.8B`](https://huggingface.co/Qwen/Qwen3.5-0.8B), revision `2fc06364715b967f1860aea9cf38778875588b17`.

Data: [`lothnic/themis-decision-mixture`](https://huggingface.co/datasets/lothnic/themis-decision-mixture), revision `92843bdf66de205d6b4238c6a0d70701fd6737a7`. Source datasets, attribution, and source-specific terms are documented in the dataset card.

| Split | Decisions | Packed requests |
|---|---:|---:|
| Train | 44,577 | 33,573 |
| Validation | 3,170 | 2,275 |
| Calibration | 2,606 | 1,945 |
| Test | 3,693 | 2,781 |

The training split contains 30,400 choices, 7,292 yes/no decisions, and 6,885 ordered scores; 12,900 decisions have soft probability targets. Labels, categories, and provenance are excluded from model inputs. Choice options and their targets are permuted together during training. Yes/no and ordered-score ordering is preserved.

Training ran for three epochs, finishing at 6,297 optimizer steps on one A100 40 GB. The published model is the **earlier checkpoint at step 4,153**, which had the lowest validation soft cross-entropy, **0.656687**. The final step's validation loss was higher, so completing training did not make the final weights the selected release.

| Setting | Value |
|---|---|
| Precision | BF16 base, FP32 LoRA parameters, BF16 CUDA autocast |
| LoRA rank / alpha / dropout | 32 / 64 / 0.05 |
| Optimizer | AdamW, weight decay 0.01 |
| Learning rate | 1e-4, cosine decay to 1e-5, 3% warmup |
| Effective batch | 16 packed requests |
| Objective | Soft cross-entropy, normalized by decisions per accumulation window |
| Gradient clipping | 1.0 |
| Gradient checkpointing | Enabled |
| Seed | 3407 |

One positive temperature per decision type was fitted on the calibration split only: choice **1.157726**, yes/no **1.107598**, score **1.020533**. The benchmark evaluations below did not update model weights or refit these temperatures.

## Evaluation

All results below use checkpoint 4,153. External benchmark results are local reproductions, not operator-verified leaderboard submissions. The tasks and scoring rules differ, so the percentages should not be treated as directly comparable measures of general intelligence.

| Evaluation | Decisions | Result | Protocol |
|---|---:|---:|---|
| Mixture held-out test | 3,693 | **77.34%** | Agreement with target distribution's highest-probability option |
| Typed Decisions official test | 2,000 | **70.45%** | Agreement with stored `gold.label`; 400 cases, five questions per case |
| SysOne Bench v2.1.0 | 1,240 | **76.85%** | 953 correct decisions across 952 evaluation cases; official scorer |

### Mixture held-out test

| Metric | Raw | Calibrated |
|---|---:|---:|
| Soft cross-entropy ↓ | 0.708560 | 0.704117 |
| KL divergence ↓ | 0.393148 | 0.388705 |
| Half-vector Brier score ↓ | 0.097691 | 0.096900 |
| Argmax agreement ↑ | 77.34% | 77.34% |
| ECE on 2,134 one-hot targets, 15 bins ↓ | 4.21% | 2.28% |

The Brier score in this table is `0.5 * sum((p - target)^2)`, averaged over decisions. It must not be directly compared with reports using full-vector Brier or binary positive-class Brier.

On 256 deterministic choice examples, changing option order changed the raw top answer in **16/256 cases (6.25%)**. Mean total-variation distance was **0.06115**. The model is not invariant to option ordering.

### Typed Decisions

Dataset: [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions), revision `7189f46b02248bd2569df7325c9df8e3b24e90a7`, `all/test`. All 400 cases and 2,000 questions fit the context limit; no truncation or inference errors occurred. Accuracy uses stored labels rather than recomputing labels from rounded probability distributions.

| Workflow | Accuracy |
|---|---:|
| Invoice | 83.4% |
| Customer support | 69.4% |
| Agents | 67.0% |
| Security | 62.0% |

**This is not a zero-shot benchmark claim.** The Themis training split includes 1,747 decisions from 554 cases in this benchmark's training data. Exact case-state/ID audits found no official test cases in Themis train, validation, or calibration. However, 352 official-test decisions from 172 cases were already represented in the mixture's test split and had been evaluated previously. This limits the independence of subsequent evaluation and model-development conclusions. These checks do not rule out semantic overlap or base-model pretraining exposure.

Calibration did not improve every out-of-distribution metric: Typed Decisions soft cross-entropy rose from **0.898235** to **0.899606**, and 15-bin stored-label ECE rose from **10.33%** to **11.87%**. Confidence should be assessed on the intended deployment distribution.

### SysOne Bench

Source: [`instax-dutta/sysone-bench`](https://github.com/instax-dutta/sysone-bench), commit `93afcfe1bd450dc052b21768e2b95b20316ebb68`, corrected **v2.1.0** manifest. The manifest SHA-256 is `0d21a64c6b61b6be7b49050dbd89f89f934912a7b586734e475308e9c9422ea2`.

The benchmark has 1,190 cases in total. The headline excludes its 238 calibration cases and uses only 952 evaluation cases / 1,240 decisions. Ordered-score answers use the probability-weighted expected level, with the official nearest-level correctness rule. This differs from modal-level classification. Existing mixture temperatures are used; benchmark-fitted calibration diagnostics do not alter the headline answers.

| Suite | Accuracy |
|---|---:|
| Multilingual intent | 95.00% |
| Triage | 92.71% |
| Moderation | 91.67% |
| AG News | 81.25% |
| Guardrails | 81.25% |
| Banking77 subset | 80.21% |
| Emotion | 72.40% |
| MNLI | 54.17% |
| SST-5 | 33.33% |

Normalized whole-state matching found no exact overlap with the mixture splits. A literal-content audit flagged one evaluation case containing “Please delete my account.”, also found inside an unrelated aggression-rating training example. Excluding that case gives **952/1,239 = 76.84%**. This is a limited text-overlap check, not proof of zero contamination. Corrected labels use upstream source labels and reviewed emotion labels; no independent double-adjudication claim is made. The upstream ECE implementation differs from standard count-weighted ECE, so its values are not presented here as standard ECE.

## Intended use and limitations

Themis is intended for experimentation with routing, triage, classification, rubric-based assessment, and probabilistic decision components. Evaluate it against your own evidence, options, and error costs before relying on its answers.

- Text only: no image, document-image, bounding-box, or region outputs.
- A small decision model can choose confidently from flawed options, misread a rubric, or miss factual and contextual details. A highest-probability option is not a verified fact.
- The 2,048-token limit includes the entire request. Longer contexts and automatic evidence truncation are not validated.
- Confidence is not universally calibrated. Domain shifts, option wording, option order, and other questions in the request can alter predictions.
- This release has no demonstrated fairness, robustness, or safety guarantee. “Security”, “moderation”, and “guardrails” benchmark scores do not make it a complete security or moderation system.
- Its sentiment and natural-language-inference results show substantial room for improvement. Strong scores on short intent tasks should not be generalized to all decision tasks or all languages.
- The released weights are intended for inference. Training resume additionally requires the original optimizer, scheduler, sampler, and RNG states; those are not included here.

## Files and reproducibility

- `model.safetensors`: complete text-backbone and trained LoRA weights; no base-model network fetch at load time.
- `config.json`: architecture, codec mapping, and fitted temperatures.
- `modeling_themis.py`, `configuration_themis.py`, `codec_themis.py`: custom model and request encoder.
- Tokenizer files: the exact saved tokenizer, including `<answer>`.
- `calibration.json`, `training.json`, `evaluation.json`: settings, pinned revisions, and aggregate evaluation results.
- `release-verification.json`, `checksums.sha256`: local inference-equivalence checks and file integrity hashes.

The architecture is adapted from the [Qwen3.5 text backbone](https://huggingface.co/Qwen/Qwen3.5-0.8B), with task-specific encoding, attention behavior, and readout as described above. The base weights remain subject to their Apache 2.0 license and attribution; this release includes the upstream license and a notice identifying the modifications.