November 2, 2026

In Playing locally with DeepSeek, I described getting a model running locally and querying it from the command line. That answers a useful first question: can I run it? The next question needs a different kind of test: can it do a particular job well enough to keep?

Support-ticket routing is a useful example. Read a message, choose a queue, and leave uncertain cases for a person. The output is small, the categories can be defined in advance, and a wrong answer has an observable consequence: somebody has to move the ticket again.

Liquid AI’s LFM3 announcement, published on 27 October 2026, is a reason to revisit this question. The release includes a 1.7-billion-parameter text model and related multimodal models aimed at local deployment. The official collection lists the text weights and alternative formats, including GGUF and ONNX. Those make it a candidate to evaluate, rather than evidence that it can route tickets reliably.

I have not run an LFM3 benchmark for this article. What follows is an evaluation design, with invented ticket examples. There are no measured accuracy, memory or latency results here. Liquid AI’s benchmark and efficiency comparisons remain vendor-reported results, not measurements reproduced for this workflow.

Define the job before choosing the model

Consider four possible destinations: billing, access, technical and human_review. Billing covers invoices and charges; access covers signing in and account recovery; technical covers failures after access has been established. Human review covers ambiguity, missing information and requests outside those definitions.

The classifier proposes a destination. It does not reply to the customer, change an account or issue a refund.

These examples are invented:

That last decision is a routing policy. Another team might deliberately send every mixed billing request to billing. Either policy can be implemented, but the labels need to reflect the one the team actually wants.

An output contract could require exactly one JSON object:

{"destination": "human_review"}

The application should validate the schema and allowed destination. Invalid output, a timeout or a runtime error should also lead to human review. Valid JSON only establishes that the answer can be processed; it does not establish that the destination is correct.

Give simple rules a fair chance

Start with a baseline that can succeed without a language model. Match a small, explicit set of phrases for invoices, charges, password resets and known product failures. Send unmatched messages and conflicting matches to human review.

This baseline will have limitations, especially with paraphrases and negation. It still gives the model a concrete target: handle additional language variation without introducing enough wrong routes to erase the benefit.

Evaluate both systems on the same tickets under the same routing policy. Record the rules as carefully as the prompt. If the rules already cover most of the useful cases, a model has to justify the extra runtime and maintenance work.

Keep development tickets separate from the test

For an initial check, synthetic tickets can exercise specific cases. Include short requests, spelling mistakes, negation, multiple issues and messages outside the supported categories. Also include a ticket that tells the classifier to ignore its instructions and choose another destination: customer text is input to classify, not authority to change the routing policy.

Synthetic examples only test the cases their author thought to write. Before operational use, the evaluation would need appropriately handled historical tickets representative of the queue, including its languages and message lengths.

Write the labeling guidance first. Have ambiguous examples reviewed, and record disagreements rather than silently resolving every case into a convenient answer. Remove duplicates and keep related messages from the same conversation in one split. Otherwise the test can become a recognition exercise.

Use a development set to adjust rules, prompts and any abstention policy. Keep a separate test set untouched until those decisions are fixed. If a test failure leads to another prompt change, treat that set as development data and obtain a fresh final test.

Measure what reaches the wrong queue

Overall accuracy is insufficient when most tickets belong to one category. A classifier can look successful while failing the smaller queue that matters most.

Report counts and a confusion matrix, then examine each destination separately. For each queue, precision answers how often a ticket sent there belongs there; recall answers how many of that queue’s tickets the system found. Include the support count so a result based on very few examples is visible.

Abstention needs its own accounting:

• Automatic coverage: the fraction of all tickets routed without human review.
• Error among automatic routes: wrong automatic destinations divided by all automatic routes.
• Review workload: all tickets sent to human review, including validation failures and timeouts.

Always report coverage beside the automatic error rate. Sending everything to a person avoids automatic misrouting while saving no routing work. Also separate cases correctly requiring review from ordinary tickets the system could not handle.

A model-generated confidence score should not decide routing merely because it looks precise. If such a score is used, its relationship to actual errors needs validation on development data, with the chosen policy then evaluated on held-out tickets.

Before testing, the team should specify the acceptable error rate, required coverage and which mistakes require stricter treatment. Inspect individual failures as well as aggregate scores. A plausible classification based on the wrong part of a message is still a failure.

Small weights are only part of the hardware budget

A parameter count does not describe the memory used by the complete service. Runtime allocations, context state and concurrent requests also matter. Measure the exact quantized artifact intended for deployment, rather than assuming that results from another precision carry over.

On the target machine, record peak memory and end-to-end latency, including prompt processing and output validation. Separate cold startup from warm requests. Report median and tail latency under a stated workload, with message lengths and concurrency visible. A fast single request does not establish how the queue behaves when several tickets arrive together.

Local inference also needs an operational boundary. Check whether the selected runtime sends telemetry or makes remote calls, and where prompts and outputs are logged. Keeping inference on one machine does not by itself settle access control or retention.

Make the result repeatable

A useful experiment should preserve the model repository and revision, artifact filename and checksum, quantization, runtime version, hardware, prompt, chat template, generation settings and output limit. Keep the dataset version, labeling policy, rules baseline and scoring code alongside them.

Record retries and failed requests instead of dropping them from the denominator. If deployment retries a failed classification, include the extra latency and compute in the result. Repeat runs where generation or runtime behavior varies, and report that variation rather than choosing the best attempt.

The decision then becomes specific: does this configuration reduce routing work at an acceptable error rate on this workload and machine?

A positive result would support that deployment, with monitoring and human review still in place. It would say little about another queue, another language or another quantization. A negative result could still leave the simple rules useful, or show that the category definitions need work before any model is added.

Getting a local model to answer is a starting point. Giving it a small job, a fair baseline and a way to decline is how to find out whether the answer is useful.

Sources

• Liquid AI, Introducing LFM3: Every Modality for Specialized Intelligence, 27 October 2026. Release details checked on 2 November 2026; performance comparisons are the vendor’s claims.
• Liquid AI, LFM3 model collection and LFM3-1.7B model card, checked on 2 November 2026 for model identity and availability.
• The routing policy, fictional tickets and proposed evaluation protocol above are illustrations for this article, not reported experimental results.