LLM Foundations
From Traditional Software to LLM-Powered Systems
আপনি একটি free preview lesson দেখছেন।
For most of software engineering history, developers have built systems around a simple assumption:
Given the same input and the same state, our program should behave predictably.
We write instructions. The computer executes those instructions.
If a payment succeeds, mark the order as paid. If authentication fails, return an error. If inventory reaches zero, prevent another purchase.
The application does not decide what these rules mean. We decide.
Large Language Models introduce a fundamentally different kind of component into our software.
Instead of explicitly programming every decision, we can provide instructions and context to a model and ask it to generate an appropriate result.
That sounds like a small change.
It is not.
It changes how we think about correctness, testing, reliability, failure, observability, security, and system architecture.
Before building agents, we need a solid mental model of this change.
1. Traditional Software Is Built from Explicit Instructions
Consider a simple support-ticket classifier.
A traditional implementation might contain rules like:
public SupportCategory classify(String message) {
String normalized = message.toLowerCase();
if (normalized.contains("refund")
|| normalized.contains("charged twice")
|| normalized.contains("payment")) {
return SupportCategory.BILLING;
}
if (normalized.contains("password")
|| normalized.contains("login")) {
return SupportCategory.ACCOUNT;
}
return SupportCategory.OTHER;
}
The application follows rules written by the developer.
For a given input, we can trace exactly why a particular branch was selected.
Conceptually:
Input
│
▼
Application Logic
│
├── Rule A
├── Rule B
├── Rule C
│
▼
Output
Suppose the input is:
I was charged twice for my subscription.
The application finds charged twice and returns:
BILLING
Nothing interpreted the meaning of the sentence.
We created rules that happened to recognize certain pieces of text.
This approach has an extremely important property:
the behavior is explicitly encoded in software.
2. Deterministic Does Not Mean Simple
Traditional software is often described as deterministic.
That does not mean traditional systems are easy.
A distributed backend may involve:
- hundreds of services
- databases
- message brokers
- caches
- concurrent requests
- network failures
- retries
- race conditions
- external APIs
- asynchronous processing
Its overall behavior can become extremely difficult to predict.
But individual program instructions still follow defined semantics.
Consider:
int total = 40 + 2;
Java is not going to occasionally decide:
43
because it interpreted the situation differently.
Similarly:
if (balance.compareTo(amount) >= 0) {
approvePayment();
}
has explicit semantics.
Real systems can still produce unexpected results because of changing state, concurrency, bugs, external dependencies, or incomplete requirements.
The important distinction is that our application logic itself is expressed as explicit instructions.
3. LLMs Introduce a Different Programming Model
Now imagine replacing our keyword-based classifier with a Large Language Model.
Instead of writing every classification rule, we provide instructions:
Classify the following customer support message.
Available categories:
BILLING
ACCOUNT
TECHNICAL
OTHER
Customer message:
"I was charged twice for my subscription."
The model might respond:
BILLING
At first glance, the result looks identical.
But the mechanism is completely different.
We did not write:
if (message.contains("charged twice")) {
return BILLING;
}
Instead, we gave the model:
- instructions,
- some context,
- user-provided data,
and asked it to generate an appropriate output.
Conceptually:
Input
│
▼
Instructions + Context
│
▼
Large Language Model
│
▼
Generated Output
The model is performing an inference based on patterns learned during training and the context provided in the current request.
This is a major architectural shift.
4. What Is a Large Language Model?
A Large Language Model, or LLM, is a machine-learning model trained on large amounts of data to learn statistical patterns in language and other structured information.
At generation time, its fundamental operation can be simplified to:
Given everything I have received so far, what should come next?
Suppose we provide:
The capital of France is
The model evaluates possible continuations.
Conceptually:
"The capital of France is"
│
▼
LLM
│
▼
Possible next tokens:
Paris 0.96
Lyon 0.01
London 0.003
Berlin 0.001
...
These numbers are illustrative, not actual model probabilities.
The model then selects a token according to its generation configuration and continues.
After generating:
The capital of France is Paris
it predicts what should come next again.
Then again.
And again.
This process is called autoregressive generation.
We will examine tokens, probability distributions, sampling, temperature, and generation behavior properly in later lessons.
For now, remember:
An LLM generates output by repeatedly predicting what should come next based on the context available to it.
This simple idea has surprisingly powerful consequences.
5. An LLM Is Not a Database
One of the easiest mistakes to make is thinking of an LLM as a giant database containing everything it learned.
That mental model is misleading.
A database works roughly like:
Query
│
▼
Stored Data
│
▼
Matching Record
If we execute:
SELECT email
FROM users
WHERE id = 42;
we expect the database to retrieve a stored value.
An LLM works differently.
When asked:
What is the capital of Estonia?
it does not normally execute something equivalent to:
SELECT capital
FROM countries
WHERE name = 'Estonia';
against a hidden facts table.
It generates a response from patterns encoded in the model's parameters and the context supplied to it.
It may generate:
Tallinn
and be correct.
But the underlying mechanism is still generation, not database retrieval.
This distinction becomes extremely important when correctness matters.
6. Fluency Is Not Evidence of Correctness
LLMs can generate remarkably convincing text.
That creates a dangerous property:
incorrect answers can look exactly like correct answers.
Consider a hypothetical model response:
The Java Virtual Machine was originally developed in 1998 by
Alexander Thompson at Bell Labs as part of the JVM Research Project.
The sentence has:
- a person
- a year
- an organization
- a project name
- confident language
It looks authoritative.
That does not make it true.
An LLM can generate plausible information that is unsupported or incorrect. This behavior is commonly called hallucination.
Traditional applications fail too.
But the failure mode is different.
A database may return:
Connection refused
A REST API may return:
503 Service Unavailable
A Java application may throw:
NullPointerException
These failures are usually visible as failures.
An LLM can fail like this:
HTTP 200 OK
"The answer is ..."
The request succeeded technically.
The answer is simply wrong.
That distinction will affect nearly everything we build later.
7. The LLM Is a Component, Not the Application
Suppose we want to build an AI-powered customer-support system.
A common beginner mental model looks like:
User
│
▼
LLM
│
▼
Answer
Real applications usually require much more.
For example:
┌──────────────┐
│ User │
└──────┬───────┘
│
▼
┌──────────────┐
│ Application │
└──────┬───────┘
│
┌────────────┼────────────┐
│ │ │
▼ ▼ ▼
Context Business Security
Management Rules Policies
│ │ │
└────────────┼────────────┘
│
▼
┌──────────────┐
│ LLM │
└──────┬───────┘
│
▼
Generated Result
│
▼
Validation
│
▼
Application Action
The LLM is only one component.
Your application still owns:
- authentication
- authorization
- persistence
- business rules
- validation
- security boundaries
- external integrations
- retries
- timeouts
- rate limiting
- observability
- auditing
- error handling
This leads to one of the most important principles in this course:
Do not move responsibilities into the LLM simply because the LLM is capable of reasoning about them.
Capability and authority are different things.
8. Capability Is Not Authority
Imagine a banking application.
We might ask an LLM:
The customer wants to transfer €50,000.
Should this transaction be allowed?
Perhaps the model answers:
Yes.
Should the bank execute the transaction?
Obviously not based solely on that answer.
Whether the transfer is permitted may depend on:
- account ownership
- available balance
- transfer limits
- compliance requirements
- fraud checks
- authorization
- account restrictions
- transaction policies
Those rules belong to deterministic systems and trusted data sources.
The LLM may help interpret the user's intent:
"Send fifty euros to Jalisa"
into something structured:
{
"intent": "TRANSFER_MONEY",
"amount": 50,
"currency": "EUR",
"recipient": "Jalisa"
}
But the application must still decide whether the requested operation is valid and authorized.
A safer architecture looks like:
Natural Language
│
▼
LLM
│
▼
Interpreted Intent
│
▼
Deterministic Validation
│
├── Authentication
├── Authorization
├── Balance
├── Limits
└── Business Rules
│
▼
Execute / Reject
This pattern will appear repeatedly throughout the course.
9. Probabilistic Core, Deterministic Boundaries
A useful way to think about production LLM systems is:
┌─────────────────────────────────────┐
│ Deterministic Application │
│ │
│ ┌─────────────────────────┐ │
│ │ │ │
│ │ Probabilistic Model │ │
│ │ │ │
│ └─────────────────────────┘ │
│ │
│ Validation • Policies • Security │
│ State • Persistence • Observability │
└─────────────────────────────────────┘
The model provides capabilities that are difficult to implement with traditional rules:
- understanding natural language
- extracting information
- classification
- summarization
- generation
- semantic interpretation
- flexible decision support
The surrounding application provides boundaries.
For example:
LLM:
"This looks like a refund request."
Application:
"Is this customer authenticated?"
Application:
"Does this order belong to them?"
Application:
"Is the order refundable?"
Application:
"Is the requested amount valid?"
Application:
"Does this action require human approval?"
The model can participate in decisions without owning every decision.
10. LLM Output Should Be Treated as Untrusted Input
Backend engineers already understand an important security principle:
Never blindly trust external input.
Consider:
String userInput = request.getParameter("amount");
We would not assume that userInput contains a valid amount.
We parse it.
We validate it.
We enforce constraints.
LLM output deserves similar treatment.
Suppose we ask a model to produce:
{
"priority": "HIGH",
"refundAmount": 49.99
}
The fact that the model generated valid-looking JSON does not prove:
priorityis a supported value- the amount is correct
- the refund is allowed
- the order exists
- the customer owns the order
- the model understood the request correctly
So our system should conceptually behave like:
LLM Output
│
▼
Parse
│
▼
Validate Structure
│
▼
Validate Semantics
│
▼
Validate Business Rules
│
▼
Authorize
│
▼
Use Result
Later, structured outputs and tool calling will make these boundaries much easier to implement.
But the principle starts here.
11. Models Do Not Automatically Know Your Application State
Suppose your database contains:
Order ID: 8192
Customer: Sakib
Status: SHIPPED
Total: €89.00
Then you ask an LLM:
What is the status of order 8192?
The model does not magically know your database.
Your application's private state is not automatically part of the model's knowledge.
Something must provide that information.
For example:
Database
│
▼
Application
│
│ retrieves order
▼
Relevant Context
│
▼
LLM
Or later:
LLM
│
│ requests tool
▼
getOrder(8192)
│
▼
Application
│
▼
Database
│
▼
Tool Result
│
▼
LLM
That second architecture begins moving us toward agents.
But we are not there yet.
12. Models Do Not Automatically Know Current Information
Another common misconception is:
"The model knows the internet."
A model's trained knowledge and access to current external information are different things.
Suppose we ask:
What is the current temperature in Tallinn?
To answer reliably, the system needs access to a current data source.
The LLM itself is not inherently a weather service.
A production architecture might be:
User
│
▼
Application
│
▼
Weather API
│
▼
Current Weather Data
│
▼
LLM
│
▼
Natural-Language Response
Or, once we introduce tool calling:
User
│
▼
LLM
│
│ decides current weather is required
▼
Weather Tool
│
▼
Weather API
│
▼
Tool Result
│
▼
LLM
│
▼
Answer
The ability to connect models to external capabilities is one of the foundations of agent engineering.
We will build that later.
13. An LLM Is Not an Agent
This distinction matters enough to establish now.
Consider:
User
│
▼
LLM
│
▼
Response
This is an LLM-powered application.
It is not necessarily an agent.
Now consider:
User
│
▼
LLM
│
│ decides it needs information
▼
Tool
│
▼
External System
│
▼
Result
│
▼
LLM
│
│ decides what to do next
▼
Another Tool
│
▼
Result
│
▼
LLM
│
▼
Final Answer
Something fundamentally different has happened.
The model is participating in determining the execution path.
Instead of our application specifying every step:
Step A
↓
Step B
↓
Step C
↓
Step D
the system may behave more like:
Observe
│
▼
Decide
│
▼
Act
│
▼
Observe Result
│
▼
Decide Again
│
├── Act Again
│
└── Finish
That is much closer to an agentic execution model.
We will define agents much more precisely later.
For now:
Calling an LLM does not automatically make your application an agent.
14. LLM Applications Still Require Software Engineering
The excitement around LLMs sometimes creates the impression that traditional engineering principles are becoming less important.
In production systems, the opposite is often true.
Consider an agent that can:
read customer information
create support tickets
issue refunds
send emails
modify subscriptions
The model now sits near systems with real-world consequences.
Suddenly questions like these become critical:
What if the model calls the same tool twice?
What if the request times out after the operation succeeds?
What if the model produces invalid arguments?
What if the model chooses the wrong tool?
What if an external API returns 503?
What if a malicious user manipulates the model?
What if the model tries to access another customer's data?
What if execution stops halfway through a workflow?
What if the model provider is unavailable?
How do we know why the agent made a decision?
How do we reproduce a failure?
How much did this execution cost?
These are software-engineering questions.
Agent engineering does not replace backend engineering.
It introduces a probabilistic component into it.
15. Traditional Workflow vs LLM-Powered Workflow
Consider processing a customer refund request.
A traditional system might have:
Request
│
▼
Parse Request
│
▼
Validate Order
│
▼
Check Refund Policy
│
▼
Calculate Amount
│
▼
Execute Refund
Every transition is explicitly programmed.
Now imagine customers can write arbitrary messages:
Hey, the headphones arrived yesterday but the left side doesn't
work. I don't really want another pair. Can I just get my money back?
Understanding that request with hard-coded rules becomes difficult.
An LLM could help convert the natural-language request into:
{
"intent": "REQUEST_REFUND",
"reason": "DEFECTIVE_PRODUCT",
"replacementRequested": false
}
Then deterministic software continues:
Customer Message
│
▼
LLM
│
▼
Structured Intent
│
▼
Validate Customer
│
▼
Load Order
│
▼
Check Refund Policy
│
▼
Determine Allowed Actions
│
▼
Execute / Request Approval / Reject
This is often a powerful architecture.
The LLM handles ambiguity.
The application handles authority.
16. Where LLMs Are Particularly Useful
LLMs are especially useful when inputs or outputs involve ambiguity and language.
Examples include:
Classification
"This payment appears twice on my statement."
→ BILLING
Information Extraction
"Book a meeting with Nur next Tuesday at 3 PM."
→ person: Nur
→ date: ...
→ time: 15:00
Summarization
100 support messages
↓
LLM
↓
Concise incident summary
Transformation
Unstructured text
↓
LLM
↓
Structured representation
Natural-Language Interfaces
"What were our five largest orders last month?"
can potentially become interaction with application capabilities.
Decision Support
The model can analyze information and recommend an action while deterministic systems or humans retain final authority.
17. Where LLMs May Be the Wrong Tool
LLMs are powerful, but they should not be inserted everywhere.
Suppose we need:
subtotal = €100
tax = 20%
We should calculate:
BigDecimal total = subtotal.multiply(taxRate.add(BigDecimal.ONE));
not ask:
LLM, please calculate the final amount.
Similarly, if a rule is:
Customers may request a refund within 30 days.
we can implement:
boolean eligible = purchaseDate
.plusDays(30)
.isAfter(clock.instant());
We don't need a probabilistic model to determine a deterministic condition.
A useful heuristic is:
Use deterministic software when the problem can be expressed reliably with deterministic rules.
Use models where their strengths justify their uncertainty and cost.
18. The Cost Model Is Different Too
Traditional local computation may be extremely cheap.
For example:
if (age >= 18) {
...
}
The marginal cost is negligible.
Calling an external LLM introduces additional dimensions:
Input tokens
+
Output tokens
+
Model pricing
+
Network latency
+
Provider limits
An architecture that performs:
1 model call per request
can have very different economics from one performing:
15 model calls per request
Agentic systems can make this especially important because the number of model and tool calls may vary between executions.
Later we will treat:
- token consumption
- model selection
- latency
- tool calls
- retries
- execution length
as engineering concerns.
19. Model Calls Are Distributed-System Calls
When using a hosted LLM, your application typically communicates with a remote service.
Conceptually:
Java Application
│
│ HTTPS
▼
Model Provider
│
▼
Inference Infrastructure
│
▼
Response
That means many familiar distributed-system concerns still apply:
- network latency
- timeouts
- rate limits
- transient failures
- provider outages
- retries
- quotas
- authentication
- request size limits
But there is an additional complication.
With a traditional service, we often think:
Request → deterministic operation → response
With an LLM service:
Request
│
▼
Remote distributed system
│
▼
Probabilistic computation
│
▼
Generated response
So we must reason about both:
infrastructure reliability and model behavior reliability.
They are different dimensions.
A request can be operationally successful but semantically wrong.
20. Three Layers of an LLM-Powered System
A useful mental model is to separate an LLM-powered system into three layers.
Layer 1 — Model
The model provides capabilities such as:
generation
classification
reasoning
extraction
summarization
semantic interpretation
Layer 2 — AI Application Logic
This layer determines how the model is used:
prompts
messages
context
structured outputs
tools
retrieval
agent workflows
memory
model selection
Layer 3 — Software System
This is the larger production environment:
APIs
databases
authentication
authorization
queues
business logic
caching
observability
deployment
security
scaling
Conceptually:
┌────────────────────────────────────────┐
│ Software System │
│ │
│ ┌────────────────────────────────┐ │
│ │ AI Application Logic │ │
│ │ │ │
│ │ ┌──────────────────┐ │ │
│ │ │ LLM │ │ │
│ │ └──────────────────┘ │ │
│ │ │ │
│ └────────────────────────────────┘ │
│ │
└────────────────────────────────────────┘
Agent engineering eventually touches all three.
21. A Better Mental Model for Agent Engineers
Throughout this course, avoid thinking:
LLM = intelligent application
Instead think:
LLM = powerful probabilistic component
Your job as an agent engineer is to build a reliable system around that component.
That means continuously asking:
What should the model decide?
What should deterministic code decide?
What information should the model receive?
What information should it never receive?
What actions may the model request?
Which actions require authorization?
What output must be validated?
What happens when the model is wrong?
What happens when the provider fails?
How do we observe what happened?
How do we evaluate whether the system is actually improving?
These questions are much more important than learning a particular framework API.
22. Frameworks Come Later
Eventually we will use abstractions that make code look deceptively simple.
For example, later we may have something conceptually similar to:
Assistant assistant = AiServices.create(
Assistant.class,
model
);
Or:
assistant.chat("Help me solve this problem");
That convenience is useful.
But if we begin there, important architecture disappears behind the abstraction.
We first need to understand:
What request is actually being sent?
What messages are included?
What context does the model receive?
What happens to conversation history?
How is output generated?
How is structured output enforced?
How are tools represented?
Who executes a tool?
What happens when execution fails?
Once those questions have answers, frameworks become productivity tools instead of magic.
That is why this course follows the principle:
Understand the mechanism before depending on the abstraction.
23. The Engineering Shift
Traditional software engineering often asks:
What exact instructions should the computer execute?
LLM-powered software adds another question:
What decisions can safely be delegated to a probabilistic model?
That word matters:
safely.
The goal is not maximum autonomy.
The goal is the appropriate amount of autonomy for the problem.
Sometimes the correct architecture is:
LLM makes recommendation
Human makes decision
Sometimes:
LLM extracts intent
Application makes decision
Sometimes:
LLM selects a read-only tool
Application executes it
And sometimes:
Agent independently performs several actions
within carefully defined boundaries.
Agent engineering is largely about designing those boundaries well.
Practical Exercise — Decide What the Model Should Own
Imagine you are designing an AI-powered e-commerce support system.
Customers can write:
My package arrived damaged. I paid €89 for it and don't want
a replacement. Please refund me.
The system has access to:
Customer Service
Order Service
Payment Service
Inventory Service
Email Service
Consider the following responsibilities:
- Understanding that the customer wants a refund
- Identifying why the customer wants the refund
- Determining which order the customer means
- Checking whether the authenticated customer owns the order
- Checking whether the order is eligible for a refund
- Determining the actual amount previously paid
- Deciding whether company policy permits an automatic refund
- Executing the refund
- Writing a natural-language response explaining the outcome
Before continuing, classify each responsibility as primarily belonging to:
LLM
Deterministic Application
LLM + Deterministic Application
Do not optimize for using the LLM as much as possible.
Optimize for correctness, security, and appropriate responsibility boundaries.
Practical Exercise — Identify Bad Uses of an LLM
For each scenario, decide whether an LLM is appropriate and explain why.
Scenario A
Determine whether:
19 > 18
Scenario B
Determine whether this message represents a cancellation request:
I've enjoyed the service, but I'm moving abroad next month
and don't think I'll need the subscription anymore.
Scenario C
Check whether a JWT signature is valid.
Scenario D
Summarize 30 customer-support messages into a short incident report.
Scenario E
Determine whether a customer has sufficient account balance for a €500 transaction.
Scenario F
Extract a shipping address from an unstructured customer email.
The important part is not simply answering yes or no.
Explain which property of the problem makes deterministic software or an LLM the better fit.
Design Exercise — Draw the Boundary
Design a high-level architecture for this feature:
A customer writes a natural-language message asking to change the delivery address of an existing order.
Your system must:
- understand the request
- identify the intended order
- extract the new address
- verify order ownership
- determine whether the order can still be modified
- validate the address
- update the order
- explain the result to the customer
Draw something similar to:
User
│
▼
...
│
▼
LLM
│
▼
...
│
▼
Order Service
Your architecture should clearly indicate:
- where the LLM participates
- where deterministic validation occurs
- where authorization occurs
- which component performs the actual state change
There is more than one reasonable design.
The important thing is being able to justify the boundaries.
Questions You Should Be Able to Answer
Before moving to the next lesson, make sure you can answer these without memorizing definitions.
1. Why is an LLM different from a database?
A database retrieves stored data according to defined operations. An LLM generates output based on learned patterns and the context supplied to it.
2. Why can an LLM produce different outputs for apparently identical requests?
Generation is probabilistic and can involve sampling from possible continuations. We will explore this mechanism in detail later.
3. Why should LLM output be treated as untrusted input?
Because syntactically valid and confidently expressed output can still be incorrect, unsupported, unsafe, or inconsistent with business rules.
4. Should an LLM decide whether an authenticated customer may access a database record?
Normally, no.
Authorization should be enforced by deterministic application logic.
5. Why isn't calling an LLM enough to make a system an agent?
An LLM-powered application may simply perform one model request and return the response. Agentic systems introduce model participation in selecting actions and determining parts of the execution path.
6. What does "probabilistic core, deterministic boundaries" mean?
Use the model where semantic interpretation and flexible reasoning provide value, while surrounding it with deterministic validation, policies, security, state management, and other controls.
7. Why does traditional backend engineering still matter when building agents?
Agents still operate inside software systems involving networks, databases, APIs, security, failures, concurrency, persistence, observability, and business rules. LLMs introduce additional uncertainty rather than eliminating these concerns.
Key Takeaways
An LLM is not an application, database, search engine, or agent.
It is a probabilistic model capable of generating outputs from the context provided to it.
Traditional software primarily executes explicitly programmed instructions. LLM-powered systems introduce components whose behavior is generated rather than completely specified by deterministic application logic.
That capability is powerful precisely because it allows software to handle ambiguity that would be difficult to encode manually.
But it introduces new failure modes.
A production system must therefore carefully decide:
What the model can interpret
What the model can decide
What the model can request
What deterministic software must validate
What actions the model is allowed to influence
The strongest LLM-powered systems are usually not systems that give the model unlimited control.
They are systems that establish clear boundaries between probabilistic intelligence and deterministic authority.
In the next lesson, we will go deeper into what the model actually receives.
We will study tokens, tokenization, and context windows—the foundation for understanding model input limits, conversation history, memory, retrieval, latency, and eventually the economics of agent execution.