SyntaxPilot
Courses/Generative AI Engineering/Lesson 1
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Generative AI Engineering · Lesson 1 of 90

What an AI engineer builds

AI engineer vs ML engineer vs data scientistLLM applicationfoundation modelmodel providerAPI-first developmentbuild vs buythe AI engineering stack (model, data, orchestration, evaluation, deployment)how this course complements the conceptual AI coursecourse roadmap

These are the notes for Generative AI Engineering episode 1. You will see why a hand-written rules bot fails on real customer questions, why training your own model is out of reach for most teams, and what AI engineers do instead: build LLM applications on top of foundation models rented from model providers. You will make your first real call to Claude from Python, read the real JSON reply, build a real customer-reply bot, meet a real API error, and write a tiny evaluation that catches two real bugs. You will also learn the difference between a data scientist, an ML engineer and an AI engineer, and the five layers of the AI engineering stack.

Every model reply in these notes is real, copied from the terminal, not a stand-in. The model code ran with the official anthropic Python SDK 0.125.0 on Python 3.9.6, on a Mac where the API key is stored in the environment variable ANTHROPIC_API_KEY (it is never shown; you will only ever see $ANTHROPIC_API_KEY or sk-ant-…hidden). The rules bot runs as-is in the SyntaxPilot compiler (language: Python 3.13). Episode 2 shows you how to set up your own key so you can run every example yourself.

The problem: a shop drowning in questions

Meena sells sarees online from a small shop in Surat. Every day about 300 customers message her, mostly with the same questions:

  • "Can I return this?"
  • "When will my parcel arrive?"
  • "Do you take cash on delivery?"

She replies by hand, replies take hours, and customers who wait too long buy somewhere else. She asks: can a program reply for me, day and night?

Attempt 1: an if/else rules bot (fails)

The obvious idea is a list of rules: if the message contains certain words, send a certain reply.

def rules_bot(question):
    q = question.lower()
    if "return policy" in q:
        return "Returns accepted within 7 days."
    if "delivery time" in q:
        return "Delivery takes 3-5 days."
    return "Sorry, I did not understand."

questions = ["What is your return policy?",
             "Can I send the saree back?",
             "When will my parcel arrive?"]
for q in questions:
    print(q, "->", rules_bot(q))

Output:

What is your return policy? -> Returns accepted within 7 days.
Can I send the saree back? -> Sorry, I did not understand.
When will my parcel arrive? -> Sorry, I did not understand.

Two of three customers asked exactly what our rules cover, and the bot failed them.

Why it failed: rules match exact words, but customers send meaning. "Send it back", "can I get a refund?" and "refund please?" are one question to a human and three strangers to an if statement. People can say the same thing in hundreds of ways, with typos, slang and mixed languages. Adding a rule for each one creates a giant mess that still misses the next customer. We need a program that understands language: a language model.

Attempt 2: train our own language model (fails)

A model is a program that learned a skill from examples instead of being given rules. To build a language model from scratch you need:

What it needs Why it's a problem for Meena
Enormous amounts of text (a big part of the public internet) She has a few hundred chat messages
Thousands of GPUs running for weeks She has one laptop
A team of machine learning experts working for months She needs a bot this week

A GPU (graphics processing unit) is a chip that does millions of small calculations at the same time; training large models keeps thousands of them busy. Attempt 2 doesn't fail with an error message. It fails with a bill nobody can pay.

The fix: rent a foundation model through an API

You don't build a power plant to run a fan. Someone else builds one big plant, an electricity company runs it, you plug into a socket and pay for the units you use. AI works the same way today:

Electricity at home AI in your app Meaning
Power plant Foundation model a huge model trained once, on broad data
Electricity company Model provider a company that runs models and rents them out
Wall socket API the doorway your code uses to send text and get text back
Meter and bill Pay per token you pay only for what you use
  • An LLM (large language model) is a model trained on huge amounts of text to predict the next piece of text. Done very well, this lets it answer questions, summarise, translate and write code.
  • The term foundation model comes from a 2021 Stanford report, On the Opportunities and Risks of Foundation Models (Bommasani et al., arXiv 2108.07258), which describes models "trained on broad data at scale" that "are adaptable to a wide range of downstream tasks". Foundation, because many different applications are built on the same model.
  • A model provider runs these models on its own computers and lets anyone use them over the internet.
  • An API (application programming interface) is a doorway one program uses to talk to another. Your Python code sends a request through it and gets a reply back.
  • A token is a small piece of text, often part of a word. Providers count and bill by tokens. Episode 4 counts them in code.

The flow of every LLM application:

Your app ──question──▶ API (over the internet) ──▶ Provider runs the LLM ──reply──▶ Your app

How these notes talk to a real model

Piece What we use
Provider Anthropic
Model claude-haiku-5-5, the smallest and fastest model in the current line-up. Anthropic's models page (fetched 16 Nov 2026) lists it from $0.10 per million input tokens and $0.50 per million output tokens
SDK the anthropic Python library (pip install anthropic), a ready-made library from the provider that wraps the API
API key a secret password for the API, kept in the environment variable ANTHROPIC_API_KEY
  • An SDK (software development kit) saves you from writing raw web requests: you call client.messages.create(...) and it builds the request, sends it and turns the JSON reply into a Python object. Episode 3 shows the raw request underneath.
  • An API key tells the provider who is calling and who pays. Treat it like a bank card PIN: never on screen, never inside your code, never in a screenshot or a Git commit. Episode 2 shows how to store it safely.
  • An environment variable is a named value your operating system gives every program you start. anthropic.Anthropic() reads ANTHROPIC_API_KEY from the environment by itself, so the key never appears in the code.

Checking the versions used for the real runs:

$ python3 -c "import anthropic, sys; print('anthropic SDK', anthropic.__version__, '| Python', sys.version.split()[0])"
anthropic SDK 0.125.0 | Python 3.9.6

Your first real model call: hello.py

import anthropic

client = anthropic.Anthropic()          # reads ANTHROPIC_API_KEY from the environment
message = client.messages.create(
    model="claude-haiku-5-5",
    max_tokens=100,
    messages=[{"role": "user", "content": "Hello! Who are you, in one sentence?"}],
)
print(message.content[0].text)
print("stop_reason:", message.stop_reason)
print("usage:", message.usage.input_tokens, "in,", message.usage.output_tokens, "out")

Real output (python3 hello.py):

I'm Claude, an AI assistant made by Anthropic that can help with questions, writing, analysis, and everyday tasks.
stop_reason: end_turn
usage: 23 in, 44 out

How it works:

  • anthropic.Anthropic() creates a client. We never pass the key; the client reads it from ANTHROPIC_API_KEY.
  • messages.create(...) sends one request. model names the model, max_tokens caps the length of the reply (here 100 tokens), and messages is a list of messages.
  • Each message is a dictionary with a role ("user" is the person, "assistant" is the model) and its content (the text).
  • message.content[0].text is the reply's text. stop_reason: end_turn means the model finished naturally. usage says 23 tokens went in and 44 came out: these are the tokens on the bill.

What really comes back: JSON (raw_reply.py)

import anthropic, json

message = anthropic.Anthropic().messages.create(
    model="claude-haiku-5-5",
    max_tokens=60,
    messages=[{"role": "user", "content": "Say hello in five words."}],
)
d = message.model_dump(exclude_none=True)
keep = {k: d[k] for k in ["id", "type", "role", "model", "content", "stop_reason"]}
keep["usage"] = {"input_tokens": d["usage"]["input_tokens"], "output_tokens": d["usage"]["output_tokens"]}
print(json.dumps(keep, indent=2))

Real output (python3 raw_reply.py):

{
  "id": "msg_011Ch6BnYAFHxYcdhs1z1bRP",
  "type": "message",
  "role": "assistant",
  "model": "claude-haiku-5-5",
  "content": [
    {
      "text": "Hello there, friend! How are you?",
      "type": "text"
    }
  ],
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 16,
    "output_tokens": 14
  }
}
  • The reply is JSON (JavaScript Object Notation): text arranged as names and values, much like a Python dictionary. Episode 3 covers it in depth.
  • model_dump() turns the SDK's reply object into a plain dictionary; the program keeps the main fields and prints them with json.dumps(..., indent=2).
  • id identifies this one reply; role: "assistant" means the model is speaking; content is a list of blocks holding the text; stop_reason says why it stopped; usage reports the tokens you are billed for. Episode 5 goes through every field.

Your first LLM application: shop_bot.py

import sys
import anthropic

client = anthropic.Anthropic()

FACTS = ["Returns: you can return any saree within 7 days.",
         "Delivery: parcels arrive in 3-5 days."]
if "--with-cod" in sys.argv:
    FACTS.append("Cash on delivery: yes, cash on delivery is available.")

def answer_customer(question):
    system = ("You are a polite assistant for Meena's saree shop. Answer in ONE short sentence, "
              "using only these facts. If the facts do not cover the question, reply exactly: I don't know.\n"
              + "\n".join(FACTS))
    reply = client.messages.create(
        model="claude-haiku-5-5",
        max_tokens=80,
        system=system,
        messages=[{"role": "user", "content": question}],
    )
    return reply.content[0].text

if __name__ == "__main__":
    for q in ["What is your return policy?",
              "Can I send the saree back?",
              "When will my parcel arrive?"]:
        print(q, "->", answer_customer(q))

Real output (python3 shop_bot.py):

What is your return policy? -> You can return any saree within 7 days.
Can I send the saree back? -> Yes, you can return any saree within 7 days.
When will my parcel arrive? -> Parcels arrive in 3-5 days.

How it works:

  • FACTS is a plain Python list of the shop's facts. The --with-cod flag (used later in the evaluation) adds a third fact about cash on delivery.
  • The system prompt (system=...) holds instructions and background facts that steer the model: be polite, answer in one short sentence, use only these facts, and if they don't cover the question reply exactly I don't know.
  • The customer's question goes in as the user message.
  • answer_customer returns the reply's text.

The two questions that broke the rules bot now get correct answers: different words, same meaning, understood. We didn't build a model; we rented it. We wrote the instructions, the data, the call and the handling of the reply. That is an LLM application: software that uses a language model as one of its parts.

Common mistake: a plain string instead of a list of messages (wrong.py)

import anthropic

client = anthropic.Anthropic()
reply = client.messages.create(
    model="claude-haiku-5-5",
    max_tokens=80,
    messages="Can I send the saree back?",      # a plain string, not a list of messages
)
print(reply.content[0].text)

Real output (last line of the error, python3 wrong.py 2>&1 | tail -n 1):

anthropic.BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages: Input should be a valid array'}, 'request_id': 'req_011Ch6Bo1NqxnjaiLkDMjWRx'}
  • 400 Bad Request means the server received the request and rejected it as invalid. An array is a list.
  • The SDK sent the request; Anthropic's server checked it and refused it. The request_id identifies this exact request (useful when you ask the provider's support for help).
  • The rule: messages is always a list, and each message is a dictionary with role and content: messages=[{"role": "user", "content": q}].

Who builds what

In the car world In an AI team Main question
Traffic analyst Data scientist "What does our data tell us?" (analysis, statistics, charts, reports)
Engine builder ML engineer "How do we train and run a better model?" (training data, training, serving)
Taxi app maker AI engineer "How do we build a reliable product with a model that already exists?" (prompts, data, tools, tests, shipping)

In real companies these roles overlap, but the starting point differs: an AI engineer starts from the product, and the model comes ready-made.

API-first development

  1. Build the first version on a provider's API (a day, not months).
  2. Put it in front of real users: does it help?
  3. Measure quality, cost and speed.
  4. Only then optimise: a cheaper model, caching, or hosting your own model.

The old way was the reverse: collect data, train for months, and only then find out whether anyone wanted the product.

Build vs buy

Buy (provider API) Build (host or train your own)
Like buying bread from a bakery baking bread yourself
Start time minutes days to months
Cost pay per token, every time servers and people, but no per-call fee
Control the provider decides models and limits you control the model, data stays with you
Good when starting out, low or medium volume very high volume, strict privacy, special needs

Most products start by buying and later build only the parts where it pays off. This course covers both.

The AI engineering stack

Every LLM application is built from five layers (a stack is a set of layers on top of each other):

Layer What it means In Meena's bot
Model which LLM, which provider, which settings claude-haiku-5-5 from Anthropic, max_tokens=80
Data everything the model needs to know for your app FACTS, the customer's question
Orchestration code that runs the steps in order (like a conductor) answer_customer: build the system prompt and messages, call, take the reply
Evaluation tests that check whether answers are right evaluate.py below
Deployment running it for real users, reliably later in the course

Evaluation catches two real bugs

Three correct answers don't prove the bot works: 300 customers a day will ask things we never tried. Before customers see it, write a small test set: questions plus words each correct answer must contain, run against the real model.

Run 1: evaluate.py finds missing data

import sys
from shop_bot import answer_customer

tests = [("Can I send the saree back?", "7 days"),
         ("When will my parcel arrive?", "3-5 days"),
         ("Do you take cash on delivery?", "yes")]
passed = 0
for q, must_have in tests:
    ans = answer_customer(q)
    ok = must_have in ans.lower()
    passed += ok
    print("PASS" if ok else "FAIL", "|", q, "->", ans)
print(f"{passed}/{len(tests)} passed")

Real output (python3 evaluate.py):

PASS | Can I send the saree back? -> Yes, you can return any saree within 7 days.
PASS | When will my parcel arrive? -> Parcels arrive in 3-5 days.
FAIL | Do you take cash on delivery? -> I don't know.
2/3 passed

No crash, no error: only a test notices. The model isn't broken. It did exactly what we told it: use only the facts, and our facts say nothing about cash on delivery. This is a good failure: without the "reply exactly: I don't know" instruction, a model can confidently make up an answer that sounds right (episode 18 covers hallucinations). The fix belongs in the data layer.

Run 2: add the fact, and a different test fails

Running with --with-cod adds one fact to FACTS:

FACTS.append("Cash on delivery: yes, cash on delivery is available.")

Real output (python3 evaluate.py --with-cod):

PASS | Can I send the saree back? -> Yes, you can return any saree within 7 days of receiving it.
FAIL | When will my parcel arrive? -> Your parcel should arrive within 3 to 5 days.
PASS | Do you take cash on delivery? -> Yes, cash on delivery is available.
2/3 passed

Cash on delivery now passes, but the parcel test fails, a test that passed a minute ago. The answer is correct: "within 3 to 5 days". Our test wanted the exact characters 3-5 days, and this time the model wrote "3 to 5". A language model can word the same answer differently on every run. It is the rules-bot mistake again, now inside our test: matching exact words instead of meaning. Exact-match tests are brittle.

Run 3: evaluate2.py accepts every correct wording

import sys
from shop_bot import answer_customer

# Each test lists every correct way of saying the answer
tests = [("Can I send the saree back?", ["7 days", "seven days"]),
         ("When will my parcel arrive?", ["3-5", "3 to 5", "three to five"]),
         ("Do you take cash on delivery?", ["yes"])]
passed = 0
for q, allowed in tests:
    ans = answer_customer(q)
    ok = any(a in ans.lower() for a in allowed)
    passed += ok
    print("PASS" if ok else "FAIL", "|", q, "->", ans)
print(f"{passed}/{len(tests)} passed")

any(...) is True if at least one allowed wording appears in the answer. We ran it twice, because one lucky run proves little.

Real output (python3 evaluate2.py --with-cod, first run):

PASS | Can I send the saree back? -> Yes, you can return any saree within 7 days of receiving it.
PASS | When will my parcel arrive? -> Your parcel should arrive within 3 to 5 days.
PASS | Do you take cash on delivery? -> Yes, cash on delivery is available.
3/3 passed

Second run:

PASS | Can I send the saree back? -> Yes, you can return any saree within 7 days of receiving it.
PASS | When will my parcel arrive? -> Your parcel should arrive in 3-5 days.
PASS | Do you take cash on delivery? -> Yes, cash on delivery is available.
3/3 passed

3/3 both times, even though the wording changed between runs ("within 3 to 5 days", then "in 3-5 days"). We found two bugs before any customer did, missing data and a test that was too strict, and fixed both without touching the model. That is the AI engineer's daily job; later episodes build much stronger evaluations.

Common mistakes (and the real error messages)

Mistake What you see (real) Why Fix
messages="Can I send the saree back?" anthropic.BadRequestError: Error code: 400 - ... 'message': 'messages: Input should be a valid array' the API takes a list of messages, not a plain string messages=[{"role": "user", "content": q}]
if "return policy" in q: for every question Sorry, I did not understand. rules match words, not meaning let a language model read the question
No cash-on-delivery fact in FACTS "FAIL | Do you take cash on delivery? -> I don't know." the model was never given the fact fix the data layer, then re-run the tests
must_have = "3-5 days" "FAIL | When will my parcel arrive? -> Your parcel should arrive within 3 to 5 days." models word answers differently each run; exact-match tests are brittle accept every correct wording, run the tests more than once
Shipping after a few good answers nothing, until a customer gets a wrong reply wrong answers don't crash keep a test set; run it after every change

How this course fits with the AI course

AI Full Course (on this channel) This course: Generative AI Engineering
how models work inside: learning from data, neural networks, transformers building applications with models: Python, APIs, prompts, RAG, agents, evals, shipping
like learning how an engine works like driving, and building a taxi service

Take either first. Whenever we need an idea from inside the model, we explain it in plain words here.

Course roadmap (90 episodes)

Part Episodes What you learn
Foundations 1-11 Python setup, HTTP and JSON, tokens, API calls, sampling, chat history, streaming, errors and retries, async, providers
Prompting 12-18 good prompts, few-shot, reasoning, chaining, templates, self-critique, hallucinations
Structured output and tools 19-24 JSON output, schemas and Pydantic, extraction, function calling, building tools
Embeddings and vector search 25-30 embeddings, similarity, vector search, ANN indexes, vector databases
RAG 31-43 answering from your own documents: loading, chunking, citations, hybrid search, reranking, evaluation, production
Agents and MCP 44-54 the agent loop, planning, memory, multi-agent systems, MCP, frameworks
Evals, observability and safety 55-64 test sets, LLM-as-judge, tracing, guardrails, prompt injection
Production 65-83 cost, latency, caching, open models, fine-tuning, multimodal, deployment, LLMOps
Interviews and projects 84-90 interview prep, three projects and a capstone

Mini project

Run this after episode 2, once your own key is in ANTHROPIC_API_KEY and pip install anthropic is done.

Extend Meena's bot:

  1. Add two new facts to FACTS in shop_bot.py (for example gift wrapping and a size chart).
  2. In evaluate2.py, add one test that should PASS and one that should FAIL (a question the facts don't cover, which should get I don't know.). Run it twice.
  3. Fix the failing one in the data layer and run the tests twice more.
One possible solution
# shop_bot.py
FACTS = ["Returns: you can return any saree within 7 days.",
         "Delivery: parcels arrive in 3-5 days.",
         "Gift wrapping: available for Rs 30.",
         "Size chart: shown on every product page."]

# evaluate2.py: two extra tests
tests += [("Can I get gift wrapping?", ["rs 30", "30"]),
          ("Do you ship to Dubai?", ["dubai"])]   # FAILS (the bot says "I don't know.") until you add a shipping fact

Then add a fact such as "International shipping: we ship to Dubai in 10 days." and re-run. The exact wording of the replies will differ on your machine; that is why each test lists allowed wordings instead of one exact sentence.

Practice

  1. In your own words, why can't if rules handle real customer messages?
  2. Name the four parts of the electricity analogy and their AI equivalents.
  3. Which layer of the stack does each part of answer_customer belong to?
  4. Run hello.py three times with your own key. Does the text change? Do the usage numbers change?
  5. In wrong.py, change messages= to the correct list. What do you get now?
  6. Change the system prompt's fallback to "Let me ask Meena and get back to you." and update the cash-on-delivery test (without --with-cod) so it checks for that fallback.
Solutions and hints
  1. Rules match exact words; people express the same meaning in endless different words, so rules never cover them all.
  2. Power plant = foundation model, electricity company = model provider, wall socket = API, meter and bill = paying per token.
  3. FACTS and the question are the data layer; building system and messages, calling client.messages.create and returning reply.content[0].text is orchestration; claude-haiku-5-5 with its settings (max_tokens=80) is the model layer.
  4. The wording can change from run to run (a model does not give the same sentence every time). The input token count stays the same for the same prompt; the output count changes with the length of the reply.
  5. messages=[{"role": "user", "content": "Can I send the saree back?"}]. The call now succeeds and prints a reply. There is no system prompt, so the model knows nothing about Meena's shop: expect a general answer (for example "it depends on the shop's return policy") rather than "7 days". Another reason the data layer matters.
  6. Replace reply exactly: I don't know. with reply exactly: Let me ask Meena and get back to you. and use ("Do you take cash on delivery?", ["ask meena"]). Run it twice.

Quick quiz

A customer asks "Is delivery free?" and the bot gives a wrong answer. In which layer do you fix it?

  • A. Train a new model from scratch
  • B. Data: add the delivery-charge fact
  • C. Delete the evaluation tests
  • D. Deployment: use a faster server
Answer

B. The model was never given the fact. Add it to the data layer, then re-run the tests to prove the fix. Training a new model would be building the power plant all over again.

Interview questions

  • What is the difference between an AI engineer, an ML engineer and a data scientist?
  • What is a foundation model? Why "foundation"?
  • What is an LLM application? Give an example you use.
  • What does API-first development mean, and why is it popular for LLM products?
  • When would you build (self-host) instead of buy (use a provider API)?
  • Name the layers of the AI engineering stack and give one example task in each.
  • What is a system prompt, and what did Meena's system prompt do when a fact was missing?
  • Why can an exact-match test fail on a correct LLM answer, and how do you make it robust?
  • Why are confident wrong answers more dangerous than crashes, and how do you catch them?
  • Where should an API key live, and why never in the code?

Glossary

Term Meaning Example
Model a program that learned a skill from examples instead of rules a language model
Language model a model that works with text answers "Can I send it back?"
LLM (large language model) a language model trained on huge amounts of text to predict the next piece of text claude-haiku-5-5
Foundation model a model trained on broad data at scale, adaptable to many tasks one LLM used for support bots, coding help and translation
Model provider a company that runs models and rents access to them Anthropic
API (application programming interface) a doorway one program uses to talk to another the Claude Messages API
SDK (software development kit) a ready-made library from the provider that wraps the API pip install anthropic
Client the SDK object that sends your requests anthropic.Anthropic()
API key a secret password that identifies who calls the API and who pays; never shown or committed $ANTHROPIC_API_KEY, shown as sk-ant-…hidden
Environment variable a named value the operating system gives every program you start ANTHROPIC_API_KEY
Token a small piece of text that models read and providers bill by usage: 23 in, 44 out
max_tokens the most tokens the model may write in its reply max_tokens=100
GPU (graphics processing unit) a chip that does many calculations in parallel; used to train and run models thousands are used to train an LLM
JSON a text format of names and values, like a Python dict {"role": "assistant"}
Message one turn in a conversation: a role plus content {"role": "user", "content": "Hi"}
Role who is speaking in a message: user or assistant "user"
System prompt instructions and background facts that steer the model, passed as system= "You are a polite assistant for Meena's saree shop..."
stop_reason why the model stopped writing end_turn = finished naturally
Usage the input and output token counts of one call "input_tokens": 16, "output_tokens": 14
400 Bad Request the server rejected the request as invalid messages: Input should be a valid array
Request id a unique id the provider gives each request, for debugging and support req_011Ch6Bo1NqxnjaiLkDMjWRx
LLM application software that uses a language model as one of its parts Meena's reply bot
Data scientist finds answers and insights in data "which products sell in winter?"
ML engineer trains models and runs them efficiently trains a recommendation model
AI engineer builds products on top of existing models builds the reply bot
API-first development build the first version on a provider's API, optimise later a working bot in a day
Build vs buy host or train your own model vs use a provider's bake bread vs buy bread
AI engineering stack the five layers of an LLM app: model, data, orchestration, evaluation, deployment Meena's bot has all five
Orchestration code that runs the steps of an LLM app in order answer_customer
Evaluation tests that check whether the answers are right evaluate.py
Test set questions with expected answers (or must-have words) tests = [...]
Exact-match test a test that wants one exact string; brittle with LLMs "3-5 days" in ans
Deployment running the app for real users, reliably the bot live on the shop's chat

Summary

  • Rules match words; customers send meaning. Language models understand meaning.
  • Training your own large model needs huge data, thousands of GPUs and an expert team, so most teams rent a foundation model from a model provider through an API and pay per token.
  • A real call is about six lines with the anthropic SDK; the client reads the key from ANTHROPIC_API_KEY, so it never appears in code. The reply is JSON with content, stop_reason and usage.
  • An LLM application is your code around the model: a system prompt with instructions and facts, the messages, the call and handling the reply.
  • messages must be a list; a plain string gets a real 400: messages: Input should be a valid array.
  • Data scientists study data, ML engineers build models, AI engineers build products on models. Work API-first, buy before you build.
  • Every app has five layers: model, data, orchestration, evaluation, deployment. Our tiny evaluation caught missing data (I don't know.) and a brittle exact-match test ("3 to 5 days"); allowing every correct wording gave 3/3 on two runs.
  • Next episode: Python setup for AI engineering (venv, pip, uv, .env files and API keys), ending with your own first real call.