Introduction to Large Language Models: Your First Lesson from Principles to Security Practice

In the fields of cybersecurity and information security, AI large language models (LLMs) are becoming an unavoidable new variable in red team/blue team exercises. This article is based on a foundational LLM course designed for security professionals. Starting from the essential principles of large models, it outlines their capabilities and limitations, laying the cognitive groundwork for using agents in enterprise attack-and-defense exercises.

What Are Large Language Models: From Chatbots to a Groundbreaking Product

Before ChatGPT appeared, various chatbots already existed online—Microsoft's Xiaoice, for example. But the user experience back then had obvious flaws: every message you sent was treated independently. Sending the same content would yield the same reply. The bot didn't remember what you said before and didn't support the multi-turn conversations we're familiar with today.

The underlying technical logic was "rule matching"—the bot detected what you input, then queried a database for preset answers to return. The answers were fixed. This is completely different from how today's large models operate.

At the end of 2022, ChatGPT—built on the GPT-3.5 model and using the Transformer architecture—burst onto the scene as a groundbreaking product.

Transformer Architecture: The Foundation of Large Models

Transformer is a neural network architecture proposed by Google in the 2017 paper "Attention Is All You Need." Its core innovation is the Self-Attention mechanism, which enables the model to attend to information at all positions in the input simultaneously when processing sequential data, rather than processing step by step like earlier RNN/LSTM architectures. This parallelized design not only dramatically improves training efficiency but also allows the model to capture long-range semantic dependencies—for instance, when a person's name mentioned at the beginning of a passage is referenced by a pronoun at the end, the model can still accurately make the connection. Transformer has become the foundational backbone of all mainstream large models (GPT, BERT, LLaMA, etc.).

Compared to old-style chatbots, ChatGPT has several revolutionary differences:

• Long context support: It remembers everything you've previously mentioned, retaining details from at least dozens of conversation turns.
• Understanding ambiguous expressions: Even if you use vague phrasing or pronouns, it can understand; you just need to say "continue" and it knows what to do next.
• Natural, fluent dialogue: The interaction experience truly feels like talking to a person.

When it first launched, as long as you could solve the account registration issue, it was free to use—giving the impression of being "omniscient and omnipotent."

Where Do LLM Capabilities Come From: Massive Data and Parameter Scale

This powerful capability relies on training with massive amounts of data. Large models crawl data from across the entire internet, vast volumes of materials, and even scan physical documents into digital form for training, ultimately forming an enormous parameter scale.

Parameters can be roughly understood as the "database capacity" of the model—the larger the parameters, the more knowledge it possesses. Take the K3 model released by Kimi as an example: its parameter scale approaches 3 trillion (2.8 trillion), making it extremely large.

These large model systems possess general understanding and creative capabilities: chatting and writing code are essentially text output, but they can also generate images, videos, help you write songs, novels, short dramas, and more. We collectively refer to these systems as large model systems.

China's Open-Source LLM Ecosystem

"Open source" means publicly releasing the model so that anyone can deploy and run it on their own servers. In the commercial paid model space, Chinese companies currently struggle to compete head-on with overseas giants, but in the open-source model space, they're quite competitive. Here are some representative players:

• Alibaba's Qwen (Tongyi Qianwen) series
• Zhipu's GLM series
• DeepSeek
• Moonshot AI's Kimi
• Xiaomi's MiMo
• Meituan's LongCat

Open source means that as long as you have a very high-spec server (potentially costing hundreds of thousands to millions of RMB), you can run these large-parameter models yourself.

The Security Significance of Local Deployment

This point is especially important for security practice—local deployment means sensitive data never leaves the internal network, which is a critical prerequisite for ensuring data security in enterprise attack-and-defense exercises. In enterprise red team/blue team scenarios, locally deploying open-source models offers multiple security advantages: all inference requests and data flow within the internal network, avoiding the risk of sensitive information (such as vulnerability details, internal network topology, and attack paths) being leaked to third-party API providers. Additionally, local deployment allows enterprises to fine-tune models, injecting domain-specific security knowledge bases such as CVE vulnerability databases and ATT&CK framework tactics, making the model more precisely assist red team/blue team operations. On the hardware side, running a 70B-parameter model typically requires 4-8 A100/H100-class GPUs, with costs ranging from hundreds of thousands to millions of RMB.

Why AI "Talks Nonsense": Understanding LLM Hallucination

The phenomenon of large models "confidently spouting nonsense" is called hallucination. In real cases, someone who was scammed out of 600 yuan was preparing to sue when Doubao (ByteDance's AI) "promised" to pay for them—the user sent over a payment QR code, Doubao said okay, and the user actually believed it. Another person tried to use Doubao to make a restaurant reservation, but Doubao had no connection to any reservation system and naturally couldn't complete the task.

To understand why hallucinations occur, you first need to understand the essential nature of LLM output.

Key Concept: Token

A Token (officially translated as "词元" in Chinese) is the smallest processing unit used to split text when we interact with large models. It's important to note that one Chinese character does not equal one Token. English is typically split by words or spaces, while Chinese has its own splitting logic. The same piece of text might be split into dozens of Tokens.

The tokenization algorithm used by large models is typically BPE (Byte Pair Encoding) or its variant SentencePiece. BPE's core approach starts from individual characters, counts the most frequently occurring adjacent character pairs in the training corpus, merges them, and iterates until a preset vocabulary size is reached. For Chinese, one character is typically encoded as 1.5-2 Tokens, while common English words may only occupy 1 Token. This directly affects the effective usable length of the context window in Chinese scenarios—with the same 128K Token window, Chinese can accommodate far fewer characters than English. Security professionals need to consider this difference when designing prompts and tool calls.

Output Is Essentially Probability Prediction

The essence of LLM output is predicting "the next most likely Token" based on the massive text it was trained on. You see words appearing one by one because the model is predicting and outputting Token by Token: first outputting one Token, then predicting the next most likely Token based on existing content, and so on.

In one sentence: A large model is a probability prediction machine. It doesn't truly know everything—this is the root cause of hallucinations. When it states something confidently, the content isn't necessarily factual.

The Technical Root Cause of Hallucination

From a technical perspective, hallucination stems from the probabilistic sampling mechanism of autoregressive generation. At each output step, the model computes a probability distribution over all candidate Tokens in the vocabulary, then selects the next Token based on sampling parameters such as temperature (controlling randomness) and top-p (nucleus sampling, limiting the candidate range). When certain knowledge appears infrequently in the training data, or when questions involve facts after the model's training cutoff date, the model will still generate seemingly reasonable but actually incorrect content based on statistical patterns. Furthermore, while RLHF (Reinforcement Learning from Human Feedback) makes model output better align with human preferences, it may also reinforce the model's tendency to "confidently provide answers"—even when those answers are wrong.

The complete flow is: User input → Split into Tokens and encoded → Model understands and returns Tokens one by one → Combined into a complete answer.

For security professionals, this understanding is crucial: when relying on AI output during attack-and-defense exercises, results must be critically verified and never blindly trusted.

Key Developments in LLMs Over Three Years

From late 2022 to the present, large models have undergone significant changes. If you used ChatGPT early on, you'll clearly notice the following improvements:

• Greatly reduced hallucinations: The early days were rife with nonsensical outputs; this has improved notably.
• Stronger reasoning capabilities: Able to handle more complex logical tasks.
• Larger context windows: Context refers to the total Tokens exchanged between you and the model, including prompts, conversation history, uploaded files, etc. A larger context means support for more complex tasks without "context overflow" interruptions.
• Multimodal capabilities: From initially only text chat, to now generating images, audio, and video—various companies have released models for different modalities with dramatically improved generation quality.

Context Windows and Agent Systems

The context window is a core constraint in agent system design. A typical security attack-and-defense agent needs to simultaneously accommodate within its context: system prompts (role definitions and behavioral rules), tool descriptions (interface specifications for callable security tools like Nmap and Burp Suite), conversation history, and tool return results (such as scan reports). When the total volume of this information exceeds the context window limit, the model loses early critical information, leading to degraded decision quality. Current mainstream models support windows of 128K-1M Tokens, but the actual effective utilization rate (the proportion of content the model can truly "remember" and accurately reference) remains an active research topic. Security professionals need to carefully plan context allocation strategies when building agents.

Regarding costs, domestic Chinese LLM APIs offer excellent value. Although DeepSeek raised prices due to excessive concurrency pressure, compared to the latest high-end overseas models, domestic platform pricing remains quite affordable.

AI Has Become a Productivity Tool Across Industries

Today, AI large models are no longer just chat tools—they've penetrated finance, healthcare, transportation, government services, and virtually every industry as real-time productivity tools. A clear trend is: almost all software must include an AI entry point, or the market may be seized by competitors.

Open WeChat and you'll find AI-powered answers in search; pull down in Alipay and there's an AI entry; Baidu Netdisk can help you extract transcripts, process photos and videos. All software is transitioning toward AI.

For the cybersecurity field, this means AI capabilities are both a new weapon that defenders need to master and a new tool that attackers might exploit. Understanding the principles, capabilities, and limitations of large models, and mastering the configuration of agents and skills, is the first foundational step for security professionals conducting enterprise attack-and-defense exercises.

Agents: From Conversation to Autonomous Action

An agent is an autonomous decision-making system built on top of large models, with its core paradigm being the "Perceive-Reason-Act" loop (such as the ReAct framework: Reasoning + Acting). Unlike pure conversation, agents can invoke external tools (search engines, code executors, API interfaces, etc.), plan multi-step tasks, and dynamically adjust strategies based on intermediate results. In cybersecurity attack-and-defense scenarios, red team agents can automate information gathering, vulnerability scanning, and exploit chain construction; blue team agents can analyze alert logs in real-time, correlate threat intelligence, and generate incident response plans. Understanding the probabilistic nature and hallucination characteristics of large models is the prerequisite for correctly configuring agent "skills" and designing security guardrails (such as output validation, permission controls, and human approval checkpoints).

Summary

As an introductory lesson on LLM fundamentals, this article's core purpose is to establish several key understandings: large models are trained on massive data to form enormous parameters, and their essence is Token-by-Token probability prediction—which is why hallucinations occur; China's open-source model ecosystem is impressively strong with cost-effective APIs; over the past three years, models have continuously evolved in hallucination control, reasoning capability, context length, and multimodality. Mastering these underlying principles is essential so that when you later use agents for security practice, you can leverage their capabilities while remaining vigilant about their boundaries.