1
00:00:01,000 --> 00:00:04,810
This accompanies our article on integrating language models.

2
00:00:05,080 --> 00:00:08,570
A convincing answer does not establish a good model choice.

3
00:00:09,080 --> 00:00:11,610
Compare the work needed to complete the task.

4
00:00:12,040 --> 00:00:16,010
A fictional maintenance report describes water under a pump housing.

5
00:00:16,440 --> 00:00:19,930
A model may propose an asset reference and problem description.

6
00:00:20,360 --> 00:00:23,530
It cannot decide urgency or schedule an engineer.

7
00:00:24,120 --> 00:00:28,330
Give suitable options the same normal, incomplete and contradictory

8
00:00:28,440 --> 00:00:29,050
inputs.

9
00:00:29,560 --> 00:00:33,090
Include source text that appears to issue instructions.

10
00:00:33,320 --> 00:00:36,410
Define the expected checks before running the trial.

11
00:00:36,920 --> 00:00:41,410
Specify permitted processing, output fields and missing values.

12
00:00:41,640 --> 00:00:45,810
Record model identification, instructions and configuration.

13
00:00:46,040 --> 00:00:50,090
Explain settings that cannot be matched, so differences remain visible

14
00:00:50,120 --> 00:00:51,050
in the assessment.

15
00:00:51,560 --> 00:00:55,210
Correctly structured fields may contain incorrect information.

16
00:00:55,640 --> 00:00:58,450
A missing asset reference stays unknown.

17
00:00:58,680 --> 00:01:03,210
When two references conflict, the model must not choose an asset by guessing.

18
00:01:03,640 --> 00:01:08,770
A refusal, invalid output or timeout needs a bounded recovery route.

19
00:01:09,000 --> 00:01:13,210
Each task keeps a stable reference, so retries do not silently

20
00:01:13,240 --> 00:01:15,370
create several review items.

21
00:01:15,640 --> 00:01:19,530
Measure total waiting time, including validation and retries.

22
00:01:19,960 --> 00:01:23,130
Track human correction effort and usage costs too.

23
00:01:23,640 --> 00:01:28,090
A cheap model call does not necessarily produce a cheaply completed task.

24
00:01:28,520 --> 00:01:32,450
Choose only after running the trial against agreed requirements.

25
00:01:32,680 --> 00:01:35,370
Record limitations and appoint a maintainer.

26
00:01:35,880 --> 00:01:41,170
Relevant changes to the model, instructions or schema require a fresh assessment.

27
00:01:41,400 --> 00:01:45,330
The suitable option depends on your task and observed results.

28
00:01:45,560 --> 00:01:49,850
Our source article on LLM integration explains this evaluation

29
00:01:49,920 --> 00:01:53,050
approach and the boundaries around model output.