WEBVTT

00:00:01.000 --> 00:00:04.810
This accompanies our article on integrating language models.

00:00:05.080 --> 00:00:08.570
A convincing answer does not establish a good model choice.

00:00:09.080 --> 00:00:11.610
Compare the work needed to complete the task.

00:00:12.040 --> 00:00:16.010
A fictional maintenance report describes water under a pump housing.

00:00:16.440 --> 00:00:19.930
A model may propose an asset reference and problem description.

00:00:20.360 --> 00:00:23.530
It cannot decide urgency or schedule an engineer.

00:00:24.120 --> 00:00:28.330
Give suitable options the same normal, incomplete and contradictory

00:00:28.440 --> 00:00:29.050
inputs.

00:00:29.560 --> 00:00:33.090
Include source text that appears to issue instructions.

00:00:33.320 --> 00:00:36.410
Define the expected checks before running the trial.

00:00:36.920 --> 00:00:41.410
Specify permitted processing, output fields and missing values.

00:00:41.640 --> 00:00:45.810
Record model identification, instructions and configuration.

00:00:46.040 --> 00:00:50.090
Explain settings that cannot be matched, so differences remain visible

00:00:50.120 --> 00:00:51.050
in the assessment.

00:00:51.560 --> 00:00:55.210
Correctly structured fields may contain incorrect information.

00:00:55.640 --> 00:00:58.450
A missing asset reference stays unknown.

00:00:58.680 --> 00:01:03.210
When two references conflict, the model must not choose an asset by guessing.

00:01:03.640 --> 00:01:08.770
A refusal, invalid output or timeout needs a bounded recovery route.

00:01:09.000 --> 00:01:13.210
Each task keeps a stable reference, so retries do not silently

00:01:13.240 --> 00:01:15.370
create several review items.

00:01:15.640 --> 00:01:19.530
Measure total waiting time, including validation and retries.

00:01:19.960 --> 00:01:23.130
Track human correction effort and usage costs too.

00:01:23.640 --> 00:01:28.090
A cheap model call does not necessarily produce a cheaply completed task.

00:01:28.520 --> 00:01:32.450
Choose only after running the trial against agreed requirements.

00:01:32.680 --> 00:01:35.370
Record limitations and appoint a maintainer.

00:01:35.880 --> 00:01:41.170
Relevant changes to the model, instructions or schema require a fresh assessment.

00:01:41.400 --> 00:01:45.330
The suitable option depends on your task and observed results.

00:01:45.560 --> 00:01:49.850
Our source article on LLM integration explains this evaluation

00:01:49.920 --> 00:01:53.050
approach and the boundaries around model output.