Python API

The public API is everything importable from thinkless, thinkless.providers, thinkless.llm, thinkless.tracing, thinkless.shadow, thinkless.integrations and thinkless.server. Anything with a leading underscore is internal and may change.

Engine

thinkless.Engine

Answers typed questions with the cheapest provider that is confident enough.

Providers are tried in the order given. Each provider receives, in one call, every still-open question it supports. An answer whose normalized confidence meets the question's threshold is accepted; the rest move on to the next provider. When every provider has been tried, questions that are still open come back uncertain with the best answer seen, or abstained if nobody answered.

Parameters:

Name	Type	Description	Default
providers	Sequence[DecisionProvider]	The decision cascade, cheapest first. A typical order is rules, then small local models, then a hosted decision model, then an LLM.	()
llm	LLM | None	The reasoning plane, used by :meth:generate.	None
threshold	float	Default minimum confidence to accept an answer.	0.8
thresholds	Mapping[str, float] | None	Overrides keyed by question name, or by "<question>@<provider>" for a threshold that applies to one provider only (providers calibrate differently, so the same question often needs a different bar per model). Resolution order: question@provider, then Question(threshold=...), then thresholds[question], then threshold.	None
trust_uncalibrated	bool	Accept answers from providers without calibrated probabilities (a prompted LLM) as final. Set it to False to have such answers come back uncertain instead.	True
tracer	Tracer | None	Where spans go. Defaults to a tracer with no sinks.	None
prices	PriceTable | None	Price table for cost estimates. Defaults to the bundled table, or $THINKLESS_PRICING when set.	None
on_error	Literal['continue', 'raise']	continue logs a failing provider and moves down the cascade; raise propagates the exception.	'continue'
escalation_context	bool	Tell providers that accept it (the LLM decider) which questions of the same batch are already settled, and how. An escalated question otherwise reaches the LLM stripped of its siblings, and on the support benchmark that changed answers: asked alone whether a request for a human is an injection, one model said yes to 5 of 10 such messages. The cost is one short line per settled question.	True
deadline_ms	float | None	Time budget for one decide or decide_many call. It is checked before each provider is asked: once it has passed, the remaining providers are skipped and open questions come back uncertain with the best answer so far. A provider call that is already running is not interrupted, so give hosted clients their own timeout.	None
spend_limit	SpendLimit | None	A :class:~thinkless.SpendLimit shared by every call on this engine. Once it is used up, paid providers are skipped and :meth:generate raises :class:~thinkless.SpendLimitError; rules and local models keep answering. run(max_cost_usd=...) adds a cap for one run.	None

Example

engine = Engine([Rules(), GLiNER(), Laya(), LLMDecider(llm)], llm=llm) intent = engine.decide(ticket, Choice("What does the customer want?", options=[...])) if intent.is_("refund"): ... ...

run(name, *, input=None, max_cost_usd=None, **attributes)

Open a root span for one unit of work (a ticket, a request, a task).

Everything the engine does inside the block, including decorated tool calls, becomes part of this trace.

Parameters:

Name	Type	Description	Default
name	str	Span name.	required
input	Any	Recorded on the root span when content capture is on.	None
max_cost_usd	float | None	Spend cap for this run. Past it, paid providers are skipped and generate raises :class:~thinkless.SpendLimitError, as with the engine-wide spend_limit.	None
attributes	Any	Recorded on the root span.	{}

step(name, **attributes)

Group related work under a named phase of the trace.

rule(name, value, **attributes)

Record a deterministic check made in application code, and return its value.

Example

if not engine.rule("authenticated", ticket.customer_id is not None): ... return ask_to_sign_in()

decide(state, question, *, name=None, deadline_ms=None)

Answer one question. See :meth:decide_many for batching.

decide_many(state, questions, *, label=None, deadline_ms=None)

Answer several questions about the same state.

Questions travel through the cascade together: each provider gets one call with all the open questions it supports, which is how System One models are meant to be used and what keeps LLM-only baselines fair.

Parameters:

Name	Type	Description	Default
state	State	Text, a JSON-like object, or a list of either.	required
questions	Mapping[str, Question] | Sequence[Question]	A mapping of name to question, or a sequence of questions keyed by their key.	required
label	str | None	Span name. Defaults to the joined question names.	None
deadline_ms	float | None	Time budget for this call. Defaults to the engine's deadline_ms.	None

Returns:

Type	Description
dict[str, Decision]	Decisions keyed by question name, in input order.

extract(state, fields, *, required=(), name=None, threshold=None)

Extract fields from state. Shorthand for deciding an :class:Extract.

generate(prompt, *, system=None, max_tokens=512, temperature=None, name='generate', llm=None)

Generate text with the reasoning plane and record it as an llm span.

warmup(questions=None, *, state=WARMUP_STATE, rounds=2)

Load every local model now instead of on the first request.

Parameters:

Name	Type	Description	Default
questions	Mapping[str, Question] | Sequence[Question] | None	The questions the application will ask. When given, every non-LLM provider answers them rounds times on state, so GPU kernels are specialized for the real shapes before traffic arrives. LLM providers are never called here.	None
state	State	Sample input for the warmup rounds.	WARMUP_STATE
rounds	int	Warmup passes per provider.	2

thinkless.Run

Handle for a traced run, returned by :meth:Engine.run.

set(**attributes)

Attach attributes to the run's root span (outcome, labels, ids).

summary()

Roll-up of the run. Complete once the with block has exited.

Questions

thinkless.Choice

Bases: Question

Pick exactly one option.

Options are given as a mapping of label to description, or as a plain list of labels. Descriptions matter: every provider uses them to tell options apart, so write them the way you would brief a new teammate.

Example

Choice( ... "What does the customer want?", ... options={"refund": "wants money back", "status": "asks where an order is"}, ... )

thinkless.Score

Bases: Question

Place the input on an ordered scale.

Levels are ordered from lowest to highest. The decision value is the expected level index (a float between 0 and len(levels) - 1) and the most likely level is available as Decision.level.

thinkless.YesNo

Bases: Question

A binary question. The decision value is a bool.

yes_means and no_means optionally spell out what each answer covers, which helps with questions whose boundary is subtle.

thinkless.Extract

Bases: Question

Pull typed fields out of the input.

fields maps a field name to a description. Fields listed in required must be found for the answer to count as confident; a missing optional field is simply None.

Example

Extract(fields={"order_id": "the order number, digits only"})

thinkless.questions.question_from_spec(spec, *, name=None)

Rebuild a question from :meth:Question.spec output, or from JSON sent by a client.

Parameters:

Name	Type	Description	Default
spec	Mapping[str, Any]	A mapping with kind (choice, score, yes_no or extract) and that kind's fields.	required
name	str | None	Overrides spec["name"].	None

Raises:

Type	Description
ValueError	If the kind is unknown or the fields do not validate.

Results

thinkless.Decision

Bases: BaseModel

The resolved answer to a question.

Attributes:

Name	Type	Description
name	str	The question key.
kind	Kind	The question kind.
value	Any	The answer. A label for Choice, a float for Score, a bool for YesNo and a dict of field values for Extract. None when the decision abstained.
status	Status	accepted, uncertain or abstained.
confidence	float | None	Normalized confidence in [0, 1]. None when the answering provider is not calibrated (a prompted LLM, for example).
probability	float | None	Probability of the chosen answer, when known.
probabilities	dict[str, float] | None	Full distribution over answers, when known.
level	str | None	For Score questions, the most likely level label.
threshold	float	The threshold that was applied.
provider	str | None	Name of the provider whose answer was used.
plane	Plane | None	Plane of that provider.
model	str | None	Model identifier reported by that provider.
latency_ms	float	Wall time spent on this question across all attempts. When several questions share a provider call, the call time is counted once per question.
attempts	list[Attempt]	Every provider that was tried, in order.
usage	Usage	Tokens consumed by the answering provider's call.
cost_usd	float	Estimated cost of the answering provider's call.
raw	dict[str, Any] | None	The provider's native payload for this question.
span_id	str | None	Trace span that recorded this decision.

escalated property

True when an earlier provider answered (or failed) and was passed over.

A provider that abstains, such as a rule that does not fire, hands the question on without an escalation: that is the cascade working as designed, at no cost.

is_(value)

True when the decision was accepted and equals value.

Convenient for routing code: if intent.is_("refund"): ... never fires on an uncertain answer.

summary()

Compact form used in trace attributes.

thinkless.Attempt

Bases: BaseModel

One provider's try at one question.

thinkless.Status

Bases: str, Enum

Outcome of a decision after the cascade finished.

accepted means an answer met its threshold (or came from a provider the policy trusts without calibration). uncertain means every provider answered below threshold; the best answer is returned and the application decides what to do. abstained means no provider produced an answer.

thinkless.Plane

Bases: str, Enum

Where a piece of work ran.

rule is deterministic code, model is a small decision model, llm is a generative model, tool is an action with side effects and code is application logic recorded for the trace.

Limits

thinkless.SpendLimit

A dollar budget for an engine, over its lifetime or a rolling window.

Once the budget is used up, the engine stops asking paid providers: a decision that would have reached a hosted LLM comes back uncertain with the best answer the free providers gave, and generate raises :class:SpendLimitError. Rules and local models keep answering.

Parameters:

Name	Type	Description	Default
max_usd	float	The budget.	required
window_s	float | None	Rolling window in seconds, for example 3600 for an hourly budget. None counts everything since the limit was created.	None

The check happens before each paid call, so the last call before the limit can overshoot it by the cost of one call.

Example

engine = Engine(providers, spend_limit=SpendLimit(20.0, window_s=86400))

thinkless.SpendLimitError

Bases: ThinkLessError

A generation was refused because a spend limit is used up.

Shadow mode

thinkless.shadow.Shadow

Runs a candidate engine on the same inputs as an existing system.

The existing system (the primary) keeps answering: its result is returned unchanged, its exceptions propagate, and by default the candidate runs on background threads so it adds no latency. Each shadow run appends one record to a JSONL log with both answers, whether they agree, and what each cost. thinkless shadow report turns the log into agreement, projected savings and a per-question verdict.

The primary can be anything: a function that calls an LLM, a rules engine, a human queue, or another ThinkLess engine. The same class also audits a ThinkLess engine in production: make ThinkLess the primary and an LLM-only engine the shadow, sampled at a few percent.

Parameters:

Name	Type	Description	Default
engine	Engine	The candidate. Give it its own tracer (or none): shadow runs are separate traces, marked shadow=True, so they never add to the cost of the request they shadow.	required
log	str | Path | ShadowLog	A path for the JSONL log, or a :class:ShadowLog.	required
sample	float	Share of calls to shadow, between 0 and 1. Shadowing an LLM costs what the LLM costs, so sample high-volume traffic.	1.0
background	bool	Run the candidate on worker threads. False runs it inline, which is simpler in tests and batch jobs.	True
workers	int	Worker threads for background runs.	2
max_pending	int	Queued runs beyond this are dropped and counted, so a slow candidate can never build an unbounded backlog.	256
max_cost_usd	float | None	Stop shadowing once the candidate has cost this much.	None
capture_content	bool | None	Store the input in the log. Defaults to the candidate tracer's setting. When off, only a fingerprint of the input is stored, which still lets you join records to your own data.	None
seed	int | None	Seed for the sampling, for reproducible tests.	None
name	str	Name of the root span of each shadow trace.	'shadow'

Example

shadow = Shadow(candidate, log="shadow/intent.jsonl", sample=0.2) @shadow.watch(INTENT) ... def classify(ticket: str) -> str: ... return call_existing_llm(ticket)

compare(state, question, primary, *, name=None, cost_usd=None, to_value=None)

Run primary, shadow it, and return the primary's result.

Parameters:

Name	Type	Description	Default
state	Any	The input both systems see.	required
question	Question	The question the primary answers.	required
primary	Callable[[], Any] | Engine	A zero-argument callable, or an engine (its :class:Decision is returned).	required
name	str | None	Question key. Defaults to question.key.	None
cost_usd	CostArg	What one primary call costs, or a function of its result. Leave it out when unknown; the report then shows the candidate's cost without a saving.	None
to_value	Callable[[Any], Any] | None	Turns the primary's result into an answer comparable with the question, for example lambda r: r["intent"].	None

acompare(state, question, primary, *, name=None, cost_usd=None, to_value=None) async

Async form of :meth:compare for a coroutine-returning primary.

observe(state, question, value, *, name=None, cost_usd=None, latency_ms=None)

Shadow an answer the primary already gave.

Use it where the existing decision happens somewhere you cannot wrap, or when replaying logged decisions: pass what was decided, and what it cost if you know.

watch(question, *, state=None, name=None, cost_usd=None, to_value=None)

Decorate an existing decision function so every call is shadowed.

Parameters:

Name	Type	Description	Default
question	Question	What the function decides.	required
state	Callable[..., Any] | None	Builds the shadow input from the call's arguments. Defaults to the first positional argument.	None
name	str | None	Question key. Defaults to question.key.	None
cost_usd	CostArg	What one call costs, or a function of its result.	None
to_value	Callable[[Any], Any] | None	Turns the function's result into a comparable answer.	None

Works on plain and async functions. The function's result and exceptions are passed through untouched.

flush(timeout=None)

Wait for queued shadow runs to finish.

close(timeout=None)

Finish queued runs, stop the workers and close the log.

thinkless.shadow.ShadowStats

Bases: BaseModel

Counters for one :class:Shadow.

Attributes:

Name	Type	Description
seen	int	Calls that reached the shadow.
sampled	int	Calls picked for a shadow run.
recorded	int	Shadow runs written to the log.
skipped	int	Calls left out by sampling.
dropped	int	Calls dropped because max_pending runs were already queued.
capped	int	Calls left out because the shadow spend reached max_cost_usd.
errors	int	Shadow runs that raised. They are logged, never re-raised.
spent_usd	float	What the shadow engine has cost so far.

thinkless.shadow.build_report(records, *, source='', target=0.95, min_calls=100, monthly_volume=None)

Summarize shadow records.

Parameters:

Name	Type	Description	Default
records	Iterable[dict[str, Any]]	Records from :func:read_log.	required
source	str	Where they came from, for the report header.	''
target	float	Agreement the shadow must reach, as a 95% lower bound, for a question to be called ready.	0.95
min_calls	int	Compared calls needed before any verdict.	100
monthly_volume	int | None	Decisions a month, to project the monthly saving.	None

thinkless.shadow.ShadowReport

Bases: BaseModel

The whole log, per question and in total.

thinkless.shadow.QuestionReport

Bases: BaseModel

Everything the report says about one question.

by_plane groups agreement by the plane that answered in the shadow engine. primary_by_plane does the same for the primary, when the primary is a ThinkLess engine: that is the view for auditing a live engine against an LLM.

thinkless.shadow.export_labels(records, out, *, question=None, disagreements_only=True, label_from='primary')

Write records as labeling rows that thinkless calibrate reads.

Each row has text (the logged input), label (pre-filled from label_from, or null), both answers, the shadow's confidence and agree. Correct the labels, then calibrate on the file.

Parameters:

Name	Type	Description	Default
records	Iterable[dict[str, Any]]	Records from :func:read_log.	required
out	str | Path	Output JSONL path.	required
question	str | None	Only this question.	None
disagreements_only	bool	Only calls where the two systems disagreed, which is where a person's label teaches the most.	True
label_from	str	primary, shadow or none.	'primary'

Returns:

Type	Description
int	Rows written. Records logged without content are skipped, since
int	there is no input to label.

thinkless.shadow.agree(question, a, b)

Whether two answers agree, and for Extract, which fields agree.

Returns (None, None) when either side has no answer.

Integrations

thinkless.integrations.Router

Bases: Generic[T]

Maps the answer to a question onto a destination.

routes maps answers to destinations (node names, agents, handlers). Only an accepted answer follows its route; an uncertain or abstained decision, or an answer with no route, goes to default. That default is usually the LLM path you run today, so routing never gets worse than the current behavior.

Parameters:

Name	Type	Description	Default
engine	Engine	The engine that answers.	required
question	Question	What to decide.	required
routes	Mapping[Any, T]	Answer to destination.	required
default	T	Where everything else goes.	required
state	Callable[[Any], Any] | None	Builds the question's input from whatever the router is called with. Defaults to passing it through unchanged.	None

destinations property

Every place this router can send to, for graph declarations.

pick_value(value, *, accepted)

Route on an answer made earlier, for example one stored in graph state.

thinkless.integrations.route(engine, state, question, routes, default)

Decide question on state and return the matching destination.

Example

handler = route(engine, ticket, INTENT, {"refund": refunds, "order_status": tracking}, agent)

thinkless.integrations.gate(engine, question, *, allow_when=True, on_uncertain='block', state=None, on_block=None, name=None)

Check a tool call with a decision before it runs.

The question is asked about {"tool": name, "arguments": {...}} by default, so rules can read the arguments directly and a model sees them as text. The call goes ahead only when the decision is accepted and equals allow_when.

Parameters:

Name	Type	Description	Default
engine	Engine	The engine that answers.	required
question	Question	Usually a :class:~thinkless.YesNo such as "Is this refund within policy?".	required
allow_when	Any	The answer that lets the call through.	True
on_uncertain	OnUncertain	block (the default) or allow when no provider is confident.	'block'
state	Callable[..., Any] | None	Builds the question's input from the call's arguments instead.	None
on_block	Callable[[Decision], Any] | None	Called with the decision when a call is blocked; its return value becomes the tool's result. Without it, :class:ToolBlockedError is raised.	None
name	str | None	Tool name in the input and in errors. Defaults to the function name.	None

Example

@gate(engine, YesNo("Is this refund within policy?", name="refund_ok")) ... def refund(order_id: str, amount: float) -> str: ...

thinkless.integrations.ToolBlockedError

Bases: ThinkLessError

A gated tool call was not allowed. decision says why.

thinkless.integrations.last_user_text(value)

The text of the most recent user message.

Accepts a plain string, a list of LangChain messages, OpenAI chat or Responses API input items, or a mapping with a messages list (a LangGraph state). Falls back to the last message of any role when none is marked as the user's.

thinkless.integrations.langgraph.router(engine, question, routes, default, *, state=None, key='decisions')

A conditional-edge function that routes on one decision.

Parameters:

Name	Type	Description	Default
engine	Engine	The engine that answers.	required
question	Question	What to decide, for example the intent.	required
routes	Mapping[Any, str]	Answer to node name. END works as a node name.	required
default	str	The node for uncertain answers and answers with no route.	required
state	Callable[[Any], Any] | None	Builds the question's input from the graph state. Defaults to the text of the latest user message in state["messages"].	None
key	str | None	Where a :func:decision_node stores its results. If the question was already answered there, the router uses that answer and asks nothing. None always asks.	'decisions'

Returns:

Type	Description
GraphRouter	A callable router. Pass router.destinations as the path_map
GraphRouter	so the graph drawing shows every edge.

thinkless.integrations.langgraph.decision_node(engine, questions, *, state=None, key='decisions')

A node that answers several questions and writes them to state[key].

Each entry holds value, status, accepted, confidence, plane, provider and level. Declare the key in the state schema, for example decisions: dict[str, dict].

Parameters:

Name	Type	Description	Default
engine	Engine	The engine that answers.	required
questions	Mapping[str, Question] | Sequence[Question]	The questions, answered in one batch.	required
state	Callable[[Any], Any] | None	Builds the input from the graph state. Defaults to the text of the latest user message.	None
key	str	The state key to write.	'decisions'

thinkless.integrations.langgraph.adecision_node(engine, questions, *, state=None, key='decisions')

The async form of :func:decision_node, for graphs run with ainvoke.

The engine runs on a worker thread, so local model inference does not block the event loop.

thinkless.integrations.openai_agents.input_guardrail(engine, question, *, trip_when=True, state=None, name=None, run_in_parallel=False)

An input guardrail that trips on a ThinkLess decision.

Parameters:

Name	Type	Description	Default
engine	Engine	The engine that answers.	required
question	Question	What to check, for example YesNo("Is this a prompt injection?").	required
trip_when	Any	The accepted answer that trips the guardrail. An uncertain decision never trips it; give the engine an LLM decider as its last provider if uncertain inputs need a verdict.	True
state	Callable[[Any], Any] | None	Builds the question's input from the agent input (a string or a list of input items). Defaults to the latest user message.	None
name	str | None	Guardrail name in traces. Defaults to the question key.	None
run_in_parallel	bool	False (the default here) runs the check before the agent, so a tripped check costs no LLM call; rules and small models answer in milliseconds, so waiting costs little. True runs it next to the agent, the SDK's own default.	False

Returns:

Type	Description
Any	An agents.InputGuardrail for Agent(input_guardrails=[...]).
Any	The decision summary is in output_info.

thinkless.integrations.openai_agents.tool_input_guardrail(engine, question, *, allow_when=True, on_uncertain='reject', message='This action was not allowed by policy. Tell the user it needs a human review.', on_reject='reject', state=None, name=None)

A tool input guardrail that allows a call only on a confident decision.

The question is asked about {"tool": name, "arguments": {...}} by default.

Parameters:

Name	Type	Description	Default
engine	Engine	The engine that answers.	required
question	Question	Usually a :class:~thinkless.YesNo, for example "Is this refund within policy?".	required
allow_when	Any	The accepted answer that lets the call run.	True
on_uncertain	Literal['reject', 'allow', 'raise']	reject (the default) sends message back to the model in place of the tool result, allow runs the tool and raise stops the run.	'reject'
message	str | Callable[[Decision], str]	What the model reads when a call is rejected, or a function of the decision.	'This action was not allowed by policy. Tell the user it needs a human review.'
on_reject	Literal['reject', 'raise']	reject sends the message back; raise stops the run with ToolInputGuardrailTripwireTriggered.	'reject'
state	Callable[[Any], Any] | None	Builds the question's input from the SDK's ToolInputGuardrailData.	None
name	str | None	Guardrail name in traces. Defaults to the question key.	None

Returns:

Type	Description
Any	An agents.ToolInputGuardrail for
Any	function_tool(tool_input_guardrails=[...]).

thinkless.integrations.openai_agents.route_agent(engine, input, question, agents, default, *, state=None) async

Pick the agent for a request with a decision instead of a triage LLM.

Parameters:

Name	Type	Description	Default
engine	Engine	The engine that answers.	required
input	Any	The run input, a string or a list of input items.	required
question	Question	Usually the intent.	required
agents	Mapping[Any, Any]	Answer to agent.	required
default	Any	The agent for uncertain answers, typically the triage agent with handoffs you run today.	required
state	Callable[[Any], Any] | None	Builds the question's input from input. Defaults to the latest user message.	None

Example

agent = await route_agent(engine, message, INTENT, {"refund": refunds}, triage) result = await Runner.run(agent, message)

Serving

thinkless.server.app.create_app(engine, questions=None, *, api_key=None, allow_ad_hoc=True, max_questions=64, warmup=True, model_name='thinkless')

Build the FastAPI app around an engine.

Parameters:

Name	Type	Description	Default
engine	Engine	The engine that answers every request.	required
questions	Mapping[str, Question] | Sequence[Question] | None	Questions clients can ask by name.	None
api_key	str | None	When set, every /v1 request needs Authorization: Bearer <api_key>.	None
allow_ad_hoc	bool	Accept question specs in the request as well as registered names. Turn it off to limit clients to the registry.	True
max_questions	int	Upper bound on questions per request.	64
warmup	bool	Load local models (and warm them on the registered questions) at startup rather than on the first request.	True
model_name	str	The model reported in System One responses.	'thinkless'

Example

app = create_app(engine, [INTENT, URGENCY], api_key=os.environ["THINKLESS_API_KEY"])

uvicorn mymodule:app --workers 1

thinkless.server.mcp.create_mcp_server(engine, questions, *, name='thinkless')

An MCP server with one tool per question.

Parameters:

Name	Type	Description	Default
engine	Engine	The engine that answers.	required
questions	Mapping[str, Question] | Sequence[Question]	The questions to expose.	required
name	str	Server name shown to clients.	'thinkless'

Returns:

Type	Description
Any	The SDK's server object. Call .run() for stdio, or
Any	.run("streamable-http") to serve over HTTP.

Example

create_mcp_server(engine, [INTENT, INJECTION]).run()

Providers

thinkless.providers.DecisionProvider

Bases: ABC

Answers typed questions.

Subclasses set

name: Unique name within an engine, used in traces and allowlists. plane: rule, model or llm. kinds: Question kinds the provider can answer. calibrated: Whether its probabilities are meaningful enough to threshold. Prompted LLMs are not. accepts_context: Whether answer takes a context keyword with the decisions already settled in the same batch. The engine only passes it to providers that set this. price_key: Provider id used for price lookup.

answer(state, questions) abstractmethod

Answer every question in one call where the backend allows it.

warmup()

Load weights ahead of the first call.

close()

Release resources.

thinkless.providers.ProviderResult

Bases: BaseModel

What a provider returns for one batched call.

Attributes:

Name	Type	Description
answers	dict[str, Answer | None]	One entry per question asked. None means the provider abstained on that question.
model	str | None	The model that served the call, as reported by the backend.
usage	Usage	Tokens consumed by the call.
cost_usd	float | None	Cost reported by the backend for this call. When None, the engine estimates it from the price table.
meta	dict[str, Any]	Structural details recorded on the trace span.
content	dict[str, Any]	Prompts and raw completions; recorded only when content capture is on.

thinkless.providers.Rules

Bases: DecisionProvider

Python functions that answer questions when they can.

A rule is registered for a question name and called with the state (and, if it accepts a second argument, the question). It returns an answer to settle the question with full confidence, or None to pass it to the next provider in the cascade.

Example

rules = Rules() @rules.rule("wants_human") ... def asks_for_person(state): ... return True if "real person" in state["message"].lower() else None rules.match("intent", r"\bunsubscribe\b", "cancel_subscription") rules.extract("order", order_id=r"#\s?(\d{4,6})")

rule(question)

Decorator form of :meth:add.

match(question, pattern, value, *, field=None, flags=re.IGNORECASE)

Answer value when pattern matches the state text.

Parameters:

Name	Type	Description	Default
question	str	Question name.	required
pattern	str	Regular expression searched in the rendered state, or in state[field] when field is given.	required
value	Any	The answer to return on a match.	required

extract(question, *, field=None, flags=re.IGNORECASE, **patterns)

Extract fields with regular expressions.

Each keyword maps a field name to a pattern; the first capture group (or the whole match) becomes the value. The rule answers when at least one pattern matches.

thinkless.providers.gliner.GLiNER

Bases: DecisionProvider

Answers choice and extract questions with a GLiNER2 model.

Choice questions are scored per label (the model emits an independent score for every option) and the scores are normalized into a distribution, so confidence is comparable with other providers. All choice questions of a call share one forward pass, and so do all extract questions.

Measured on an RTX 5060 laptop GPU with gliner2.5-base-v1: about 16 ms per classification and 38 ms per extraction; about 85 ms each on CPU.

Parameters:

Name	Type	Description	Default
model	str	fastino/gliner2.5-base-v1 (English, 0.2B), fastino/gliner2.5-small-v1 (74M, fastest on CPU) or fastino/gliner2.5-multi-v1 (multilingual).	'fastino/gliner2.5-base-v1'
device	str	auto, cpu, cuda or mps.	'auto'
name	str	Provider name in traces.	'gliner'
extract_threshold	float	Minimum span confidence for an extracted field.	0.5

Requires the gliner extra. Weights download on first use.

warmup()

Load the weights and run representative inferences of each kind.

The first forward passes on a GPU are several times slower than the rest while kernels initialize, and GLiNER's TorchScript parts only specialize after a couple of runs; paying for that here keeps it off the first requests.

thinkless.providers.laya.Laya

Bases: DecisionProvider

Answers choice, score and yes/no questions with Laya.

Laya is a non-autoregressive encoder (about 421M parameters for the English checkpoint) that answers every question of a call in one forward pass. Measured on an RTX 5060 laptop GPU: about 30 ms for three questions; about 650 ms on CPU.

Parameters:

Name	Type	Description	Default
model	str	Hugging Face repo of the checkpoint. convaiinnovations/laya is the English model; see the Laya project for the multilingual and fine-tuned checkpoints.	'convaiinnovations/laya'
device	str	auto, cpu, cuda or mps.	'auto'
name	str	Provider name in traces.	'laya'
max_len	int | None	Token budget for the state. Longer inputs are truncated by the model.	None

Requires the laya extra. Weights download on first use.

warmup()

Load the weights and run one tiny inference, so the first request is not slow.

thinkless.providers.SystemOne

Bases: DecisionProvider

Any server that implements POST /v1/systemone.

Works with TypeSafe's hosted Jev and with self-hosted servers that copy its wire format, such as Kev and OpenJev. All questions of one call are answered in a single request.

Parameters:

Name	Type	Description	Default
name	str	Provider name in traces.	'jev'
base_url	str	Server root, without the /v1/systemone path.	'https://api.typesafe.ai'
model	str	Model id or alias, for example jev-latest.	'jev-latest'
api_key	str | None	Bearer token. Defaults to the api_key_env variable.	None
api_key_env	str | None	Environment variable read when api_key is omitted.	'TYPESAFE_API_KEY'
price_key	str	Provider id for price lookup. typesafe for Jev, local for self-hosted servers.	'typesafe'
timeout	float	Per-request timeout in seconds.	30.0
max_retries	int	Retries on 429, 529, 5xx and connection errors, with exponential backoff that honors Retry-After.	3
client	Client | None	A preconfigured httpx.Client (tests, proxies).	None

Use :meth:jev and :meth:self_hosted for the common setups.

jev(*, api_key=None, model='jev-latest', **kwargs) classmethod

TypeSafe's hosted Jev. Reads TYPESAFE_API_KEY by default.

self_hosted(base_url, *, name='systemone', model='latest', **kwargs) classmethod

A self-hosted server such as Kev or OpenJev. Cost is recorded as 0.

thinkless.providers.LLMDecider

Bases: DecisionProvider

Answers any question kind by prompting a generative model.

Answers carry no probabilities, so the engine treats them as uncalibrated: by default they are accepted as the final word of a cascade (see Engine(trust_uncalibrated=...)).

Parameters:

Name	Type	Description	Default
llm	LLM	The model to prompt.	required
name	str	Provider name. Defaults to llm.	'llm'
max_tokens	int	Reply budget per batch, plus 48 tokens per question.	256
retry_on_truncation	bool	When a reply is cut off at the token limit and answers are missing, retry once with three times the budget. Reasoning models are the usual cause: hidden reasoning tokens count against the limit. The retry is recorded in the trace.	True

LLM backends

thinkless.llm.LLM

Bases: ABC

A generative model.

Implementations wrap one backend. provider identifies the backend in traces and in the price table (anthropic, openai, local, ...).

Args of :meth:complete: messages: Conversation turns, oldest first. system: Optional system prompt. max_tokens: Upper bound on generated tokens. temperature: Sampling temperature. None leaves the backend default; some hosted models reject the parameter entirely, and their implementations ignore it. json_mode: Ask the backend for a JSON object when it supports that.

warmup()

Load weights or open connections ahead of the first call.

close()

Release resources.

thinkless.llm.Completion

Bases: BaseModel

The result of one generation.

cost_usd is the cost the backend reported for this call (OpenRouter does). When it is None the engine estimates cost from the price table.

thinkless.llm.local.TransformersLLM

Bases: LLM

Runs a chat model in-process with transformers.

This is the zero-setup option: no server, no API key. For throughput, run the same weights behind Ollama, vLLM or llama.cpp and use :class:~thinkless.llm.OpenAICompatibleLLM instead; a serving engine is several times faster than the plain generate loop used here.

Parameters:

Name	Type	Description	Default
model	str	Hugging Face model id or local path.	'Qwen/Qwen3-1.7B'
device	str	auto, cpu, cuda, cuda:N or mps.	'auto'
dtype	str	auto picks bfloat16 on CUDA, float16 on MPS, float32 on CPU.	'auto'
enable_thinking	bool	Passed to chat templates that support a thinking switch (Qwen3 and later). Off by default: decisions and short replies do not benefit from long reasoning traces.	False

Requires the local-llm extra.

warmup()

Load the weights and generate one token, so the first request is not slow.

thinkless.llm.openai_compat.OpenAICompatibleLLM

Bases: LLM

Chat Completions client for OpenAI and compatible servers.

Covers OpenAI itself plus Ollama, vLLM, LM Studio, llama.cpp server, Groq, Together and any other server that implements /v1/chat/completions. For OpenRouter, use :class:~thinkless.llm.OpenRouterLLM, which sets the endpoint, key and reported cost for you.

Parameters:

Name	Type	Description	Default
model	str	Model id as the server knows it.	required
base_url	str | None	Server URL, for example http://localhost:11434/v1 for Ollama. None targets api.openai.com.	None
api_key	str | None	Defaults to OPENAI_API_KEY. Local servers usually accept any value.	None
provider	str	Name used in traces and for price lookup. Use ollama, vllm or lmstudio for local servers so their cost is 0.	'openai'
token_param	str | None	max_completion_tokens (OpenAI's current name) or max_tokens (what most compatible servers accept). Chosen from base_url when omitted.	None
supports_json_mode	bool	Whether to send response_format when a caller asks for JSON. If the server rejects it, the client retries once without it and stops sending it.	True
default_headers	Mapping[str, str] | None	Extra HTTP headers sent with every request.	None
extra_body	Mapping[str, Any] | None	Extra fields merged into every request body, for server-specific options.	None

Requires the openai extra.

thinkless.llm.anthropic.AnthropicLLM

Bases: LLM

Claude models via the Messages API.

Parameters:

Name	Type	Description	Default
model	str	Claude model id. Defaults to claude-opus-5. For high-volume reply generation, claude-sonnet-5 or claude-haiku-4-5 are the cheaper options.	'claude-opus-5'
api_key	str | None	Defaults to the SDK's credential resolution (ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN or an ant auth login profile).	None
effort	str | None	Optional output_config.effort (low to max). low suits short replies and decision questions. Supported on Claude Opus 4.6 and later, Sonnet 5 and Fable; Haiku 4.5 and Sonnet 4.5 reject it, so leave it unset for those.	None
fallbacks	bool | None	Server-side refusal fallbacks (fallbacks="default"): if a safety classifier declines the request, the API re-runs it on the recommended fallback model in the same call. None (the default) turns them on for the model families that document them (Opus 5 and later, Fable 5, Mythos 5) and off otherwise. Available on the Claude API and Claude Platform on AWS; pass False when routing through Bedrock, Vertex AI or Foundry.	None
client	Any | None	A preconfigured anthropic.Anthropic client (tests, proxies, custom base URL).	None

Sampling parameters are not sent: current Claude models reject temperature and control depth through effort instead.

Requires the anthropic extra.

thinkless.llm.ScriptedLLM

Bases: LLM

Returns canned responses.

Parameters:

Name	Type	Description	Default
responses	Sequence[str] | Responder	Either a list of strings returned in order (the last one repeats), or a callable (messages, system) -> str.	required
model	str	Name reported in traces.	'scripted'
latency_ms	float	Artificial delay, to make dry-run traces look realistic.	0.0

Token usage is approximated from character counts.

thinkless.llm.from_spec(spec, *, device='auto', base_url=None, reasoning=None)

Create an LLM from backend[:model].

Examples:

local (Qwen3-1.7B in-process), local:Qwen/Qwen3-4B, anthropic (Claude Opus 5), anthropic:claude-haiku-4-5, openrouter:qwen/qwen3.7-flash, openrouter:anthropic/claude-haiku-4.5, openai:<model>, ollama:qwen3:8b, vllm:<model>.

Parameters:

Name	Type	Description	Default
spec	str	Backend name, optionally followed by a colon and a model id.	required
device	str	Device for the local backend.	'auto'
base_url	str | None	Server URL for openai, ollama and vllm.	None
reasoning	str | None	off, minimal, low, medium or high; None or default keeps the model's own default. Mapped to OpenRouter's reasoning option, Anthropic's effort, and the thinking switch of local chat templates. Short decisions rarely need reasoning, and hidden reasoning tokens are billed and count against the output limit.	None

Tracing

thinkless.Tracer

Creates spans and forwards them to sinks.

Parameters:

Name	Type	Description	Default
sinks	Sequence[Any] | None	Where finished spans go. See :mod:thinkless.tracing.sinks.	None
capture_content	bool	Record inputs, outputs, prompts and tool arguments. Turn it off where traces must not hold user content; structure, timings, confidences and token counts are still recorded.	True

Sink failures are logged and never propagate into application code.

content(value)

Return value as JSON-safe data, or a marker when content capture is off.

thinkless.tracing.Span dataclass

A timed unit of work.

kind says what the span represents: run (a root), step (an application-defined phase), decide (a batch of questions), attempt (one provider call inside a decision), llm (a generation), tool (an action) or rule (a deterministic check recorded by application code).

plane says where the work ran and drives cost and latency roll-ups.

set(**attributes)

Attach attributes. Later values overwrite earlier ones.

thinkless.tracing.TraceSummary

Bases: BaseModel

What a run cost and where its time went.

Attributes:

Name	Type	Description
llm_calls	int	Every call to a generative model, whether it generated text or answered decision questions.
generation_calls	int	LLM calls made through Engine.generate.
decisions	int	Questions resolved in the run, plus deterministic checks recorded with Engine.rule.
decisions_by_plane	dict[str, int]	How many decisions each plane resolved. unresolved counts abstentions.
uncertain	int	Decisions that ended below threshold on every provider.
escalations	int	Decisions where a provider answered below threshold (or failed) and the next provider was asked.
llm_input_tokens,	llm_output_tokens	Tokens spent on generative models.
model_input_tokens	int	Tokens processed by small decision models.
cost_usd	float	Estimated spend, from the configured price table.
time_by_plane_ms	dict[str, float]	Time inside leaf work (provider attempts, generations, tools and rules) grouped by plane.

thinkless.tracing.summarize(spans)

Summarize one trace.

Parameters:

Name	Type	Description	Default
spans	Iterable[Mapping[str, Any]]	Span dictionaries of a single trace, in any order.	required

Raises:

Type	Description
ValueError	If there are no spans.

thinkless.tracing.JSONLSink

Writes one JSON Lines file per trace.

Files are named <UTC timestamp>_<root name>_<trace id prefix>.jsonl so a directory listing reads as a run log. Each line is one span; the root span is always the last line, which makes a truncated file easy to detect.

Parameters:

Name	Type	Description	Default
directory	str | Path	Output directory, created on first write.	required

path(trace_id)

Where a trace is being written, while its root span is open.

thinkless.tracing.MemorySink

Keeps finished spans in memory. Used by tests and benchmarks.

traces()

Finished spans grouped by trace id, as dictionaries.

thinkless.tracing.ConsoleSink

Prints each finished trace as a tree, when its root span closes.

Parameters:

Name	Type	Description	Default
render	Callable[[list[dict[str, Any]]], None] | None	Callable that receives the finished span dictionaries of one trace. Defaults to :func:thinkless.tracing.console.print_trace.	None

thinkless.tracing.otel.OTelSink

Forward spans to an OpenTelemetry TracerProvider.

Parameters:

Name	Type	Description	Default
tracer_provider	Any | None	The provider to export through. Defaults to the global provider configured by the application.	None
include_content	bool	Export attributes that may contain user content.	False

thinkless.tool(fn=None, /, *, name=None)

tool(fn: F) -> F

tool(*, name: str | None = None) -> Callable[[F], F]

Record calls to a function as tool spans.

Outside a traced run the function runs untouched, so decorated tools stay usable in tests and scripts.

Example

@tool ... def refund_payment(payment_id: str, amount: float) -> dict: ...

thinkless.tracing.export.iter_decisions(paths)

Every decision recorded in trace files or directories, one dict each.

Each dict has the question name and kind, the input (state, or None when content capture was off), the value, status, plane, provider, confidence and whether it escalated, plus the trace and span ids to find it again.

thinkless.tracing.export.export_trace_labels(paths, out, *, question=None, statuses=None, planes=None, limit=None, seed=0)

Write traced decisions as rows to label, in the format thinkless calibrate reads.

Rows hold text (the input), label (the decision's value when it was accepted, else null), and the decision's status, plane, provider and confidence. Decisions traced without content are skipped.

Parameters:

Name	Type	Description	Default
paths	Iterable[str | Path]	Trace files or directories.	required
out	str | Path	Output JSONL path.	required
question	str | None	Only this question.	None
statuses	Sequence[str] | None	Only these statuses, for example ["uncertain"].	None
planes	Sequence[str] | None	Only decisions settled by these planes.	None
limit	int | None	At most this many rows, sampled at random with seed.	None

Returns:

Type	Description
int	Rows written.

thinkless.tracing.export.drift_report(baseline, current, *, llm_share_points=0.1, unsure_points=0.1, answer_shift_limit=0.15, confidence_drop=0.05, min_decisions=30)

Compare decisions in two sets of traces, question by question.

A question is flagged when the share reaching an LLM rises by more than llm_share_points, when the share not accepted (uncertain or abstained) rises by more than unsure_points, when the answer distribution moves by more than answer_shift_limit (total variation distance), or when the mean confidence of calibrated answers drops by more than confidence_drop. Questions with fewer than min_decisions in either period are reported without flags.

Confidence and pricing

thinkless.confidence

Confidence normalization.

Providers disagree about what "confidence" means. Measured on the same input, Laya reports 1 - normalized entropy for choices and max(p, 1 - p) for yes/no questions, while TypeSafe documents (k * p_max - 1) / (k - 1). A threshold of 0.8 would therefore mean something different depending on which backend answered.

ThinkLess computes one confidence from each provider's probability distribution and applies thresholds to that number only. The provider's own fields are kept untouched in Decision.raw.

The formula is the normalized maximum probability, the same one TypeSafe documents for Jev::

confidence = (k * p_max - 1) / (k - 1)

It is 0 for a uniform distribution over k outcomes and 1 when all mass sits on one outcome. For a yes/no question (k = 2) it reduces to 2 * max(p, 1 - p) - 1, so a threshold of 0.8 requires p >= 0.9.

normalize_distribution(scores)

Rescale non-negative scores so they sum to 1.

Useful for providers that emit independent per-label scores (for example sigmoid outputs) rather than a softmax distribution.

from_distribution(probabilities)

Normalized confidence of a categorical distribution.

from_yes_probability(p_yes)

Normalized confidence of a binary answer given P(yes).

thinkless.pricing.PriceTable

Looks up per-token prices and turns usage into dollars.

load(path=None) classmethod

Load a price table from path, $THINKLESS_PRICING or the bundled file.

cost(provider, model, usage)

Dollar cost of usage and whether the price was known.

Local providers always cost 0 and count as known.