Reverse Engineering Google’s SynthID: What This Project Teaches Us (and How to Use It)

 As AI-generated content becomes indistinguishable from reality, companies like Google have introduced systems to keep things traceable. One of the most important of these is SynthID, a watermarking technology designed to invisibly tag AI-generated media.

But an open-source project called reverse-SynthID, created by Aloshdenny, is showing that even sophisticated watermarking systems can be analyzed from the outside.

This article explains what the project actually does, how it works in simple terms, and how you can experiment with it yourself.

What SynthID Actually Does (in simple terms)

SynthID is not a visible watermark like a logo or text overlay. Instead, it modifies content in a way that humans cannot perceive.

For images, the system works in the frequency domain. That means instead of changing pixels directly, it slightly alters patterns that only appear when you mathematically transform the image using something like a Fourier Transform.

Think of it like this:

• The image you see stays the same

• Hidden underneath, there’s a structured signal

• That signal can later be detected by specialized tools

This is much harder to remove than metadata, because it’s embedded inside the content itself.

What reverse-SynthID Discovered

The key idea behind reverse-SynthID is surprisingly simple:
If you generate enough AI images and analyze them carefully, patterns start to emerge.

The developer used a clever trick:

• Generate uniform images (for example, completely black images)

• Since the image itself contains almost no information, any remaining signal is likely the watermark

• Convert the image into frequency space

• Look for consistent patterns across multiple samples

What emerged is that the watermark is not random noise. It has structure:

• It appears at specific frequency points

• The pattern is consistent across images

• It changes depending on resolution

This means the watermark can be studied, mapped, and partially predicted.

Why This Matters

This doesn’t mean SynthID is useless. It still works well in many real-world cases.

But it reveals something important:

• Any system that produces consistent outputs can be reverse engineered

• Even invisible signals leave detectable traces

• Security based only on secrecy is never permanent

In practice, the project shows that it may be easier to:

• disrupt detection

• or confuse classifiers

than to completely remove the watermark.

How reverse-SynthID Works (high level)

The project relies on classic signal processing rather than AI.

Main steps:

1. Generate a dataset of AI images

2. Apply a Fourier Transform (FFT)

3. Identify repeating frequency components

4. Isolate the watermark signal

5. Analyze or manipulate it

No neural networks, no proprietary access—just math and observation.

How to Use reverse-SynthID (Practical Guide)

If you want to try it yourself, here’s a simple walkthrough.

Clone the repository

Open your terminal and run:

git clone https://github.com/aloshdenny/reverse-SynthID.git
cd reverse-SynthID 

Install dependencies

The project typically uses Python with scientific libraries:

pip install -r requirements.txt

If there’s no requirements file, you’ll likely need:

• numpy

• scipy

• matplotlib

• pillow 

You’ll need AI-generated images (for example from:

• Gemini

• Imagen

• other generative tools)

Tip*Use simple images (solid colors or minimal detail) to make the watermark easier to detect.

Run frequency analysis

The core of the project is applying FFT to images.

Example (conceptually):

import numpy as np
from PIL import Image

img = Image.open("image.png").convert("L")
data = np.array(img)

fft = np.fft.fft2(data)
fft_shift = np.fft.fftshift(fft)

This transforms the image into frequency space, where hidden patterns become visible.

Look for consistent patterns

Repeat the process across multiple images and compare results.

What you’re looking for:

• bright points in the same positions

• structured grids or patterns

• signals that persist across images

These are likely parts of the watermark.

Experiment with removal (advanced)

Once identified, you can try:

• masking specific frequencies

• adding noise

• compressing or resizing

This doesn’t always remove the watermark completely, but it can degrade detection.

Limitations You Should Know

Before jumping to conclusions, it’s important to be realistic.

• This is not a full SynthID breaker

• Results depend heavily on the dataset

• Detection systems may still work even if the signal is altered

• Real-world robustness is still debated

In short:
You’re exploring behavior—not defeating the system entirely.

reverse-SynthID is a great example of how open-source research can challenge assumptions about AI safety systems.

It shows that:

• even invisible mechanisms can be studied

• simple mathematical tools remain incredibly powerful

• transparency and robustness must go hand in hand

Meet Libretto The Ultimate Toolkit for Web-Integrated AI Agents

If you’ve been building AI-driven coding agents, you know the biggest hurdle isn't writing the code—it’s getting the agent to understand the "real world" of a live website. Standard scrapers and headless browsers often overwhelm LLMs with too much noise or fail to capture how a site actually behaves.

Enter Libretto, a lightweight, robust toolkit designed to bridge the gap between your coding agent and the live web.

What is Libretto?

Libretto is more than just a browser automation tool; it’s a specialized interface that gives your AI agent a "live eye" and an efficient CLI. It allows developers to build integrations that don’t just read static data but interact with dynamic web environments in real-time.

Key Features That Change the Game:

• Token-Efficient Inspection: Stop wasting your context window. Libretto allows you to inspect live pages with minimal overhead, ensuring your agent gets the vital info without the "DOM clutter."
• Network Traffic Capture: Want to know how a site actually talks to its server? Libretto captures network traffic, making it easy to reverse-engineer and decode site APIs for more reliable integrations.
• Action Recording & Playback: You can record complex user actions and instantly turn them into automation scripts. It’s like teaching your agent a routine it can repeat perfectly every time.
• Interactive Debugging: When a workflow breaks, you don't have to guess why. Libretto lets you debug interactively on the actual site, seeing exactly where the logic hit a wall.

Why It Matters

For developers building AI agents for healthcare, finance, or e-commerce, the ability to handle "brittle" web interfaces is a superpower. Libretto simplifies the "dirty work" of web integration so you can focus on the core logic of your AI.

Mirror GitHub Repository to Azure DevOps Repos via Pipeline

Create a PAT in Azure DevOps

1. Azure DevOps → top right User Settings → Personal Access Tokens
2. Click New Token:
   • Name: mirror-pat
   • Scope: Code → Read & Write
3. Copy the generated token

Step — Add PAT as a Secret Variable in the Pipeline

1. Azure DevOps → Pipelines → select your pipeline → Edit
2. Click Variables → New variable:
   • Name: AZURE_PAT
   • Value: paste your PAT
   • Check: Keep this value secret
3. Save

Step — Add azure-pipelines.yml to your GitHub repo
Example configuration:

trigger:
  branches:
    include:
      - main

pool:
  vmImage: 'ubuntu-latest'

steps:
- checkout: self
  persistCredentials: true

- script: |
    git push --force https://$(AZURE_PAT)@://azure.com<ORG>/<PROJECT>/_git/<REPO> HEAD:main
  displayName: 'Mirror to Azure Repos'

Replace <ORG>, <PROJECT>, and <REPO> with your specific values.

How it works

1. GitHub repo (source): Pushing to main triggers the pipeline.
2. Azure Pipelines: Clones the GitHub repo and uses git push --force to copy commits.
3. Azure Repos (mirror): Receives the exact copy of the repository.

• GitHub is the source of truth.
• Azure Repos acts as a read-only mirror.

Troubleshooting

• Authentication failed: Usually caused by an incorrect $(AZURE_PAT) variable.
• Rejected push: Ensure the --force flag is included in the git push command.
• Command not found: Often a syntax error in the YAML script block.

Notes

• --force is only safe if no one is pushing changes directly to Azure Repos.
• The only URL required is the destination Azure Repos URL.

Writing Clean Specs with Control Gates

A Control Gate is a validation block at the start of your function or test suite. It’s a "fail-fast" mechanism. Instead of letting a process run with bad data, you slam the gate shut immediately if the conditions aren't met.

When we write specs using this pattern, we aren't just testing the "happy path"; we are defining the "integrity boundaries" of our logic.

The Angular/TypeScript Way

In Angular, we often use Guards for routes, but we can apply the "Gate" logic inside our component specs using Jasmine or Jest.

it('should not process payment if the gate (balance) is closed', () => {
  const account = { balance: 0 };
  
  // Control Gate
  if (account.balance <= 0) {
    expect(processPayment(account)).toBeFalse();
    return; // Gate closed
  }

  // Rest of the logic...
});

 The React/JSX Way

React developers love functional purity. Using a gate in your testing (with Vitest or Jest) ensures your hooks or components don't trigger unnecessary side effects.

test('gate prevents submission if user is unauthorized', () => {
  const user = { loggedIn: false };

  // The Control Gate
  const canSubmit = user.loggedIn ? true : false;

  expect(canSubmit).toBe(false);
  // Logic stops here—the gate worked.
});

The PHP (Pest/PHPUnit) Way

Even in the backend, the pattern is identical. Whether you are using Laravel or raw PHP, a Control Gate keeps your business logic from "bleeding" into invalid states.

test('control gate blocks invalid email formats', function () {
    $email = "invalid-at-email.com";

    // Control Gate
    if (!filter_var($email, FILTER_VALIDATE_EMAIL)) {
        $isValid = false;
    }

    expect($isValid)->toBeFalse();
});

---

Why this matters

Whether you’re writing expect(), $this->assertEquals(), or assert.strictEqual(), the architecture of your thought process is what counts.

The "Control Gate" approach gives you:

1. Readability: Anyone can see the "entry requirements" immediately.
2. Safety: You prevent deep-level logic from executing when the baseline data is trash.
3. Portability: As you saw above, you can take this mental model from a React frontend to a PHP backend without losing a beat.

Build a gate, check the ID, and only then let the logic through.

Building a Simple Agentic Workflow in Python: A Practical Guide

Agent-based architectures are becoming increasingly popular in modern software design, especially in AI-driven systems. In this article, we’ll walk through a simple yet powerful example of how to design and implement an agentic workflow in Python, using modular components and clear separation of responsibilities.

What Is an Agentic Workflow?

An agentic workflow is a system where multiple specialized components (agents) collaborate to accomplish a task. Each agent has a defined role, and data flows between them in a structured way.

Think of it as a team:

• One agent gathers information

• Another processes it

• Others could analyze, validate, or store it

Step 1: Defining Agent Blueprints

We begin by creating a module (agent_definitions.py) that contains our agent classes. These classes act as reusable blueprints.

Base Agent Class

At the core, we define a generic Agent class:

• It assigns a name to each agent

• It enforces the implementation of an execute() method

This ensures consistency across all agents.

Specialized Agents

We then create two concrete agents:

DataFetchingAgent
Responsible for retrieving data
Simulates fetching user information from a data source
Returns either valid user data or an error

DataProcessingAgent
Takes fetched data as input
Extracts specific fields such as name and occupation
Handles errors gracefully

This separation of concerns makes the system flexible and easy to extend.

Step 2: Instantiating Agents

In main_workflow.py, we import our agent classes and create specific instances:

• A fetcher configured with a data source

• A processor configured with fields to extract

Each instance is tailored for its role, demonstrating how agents can be customized dynamically.

Step 3: Implementing the Workflow

The workflow is defined as a sequence of steps inside a function.

First, fetch data. The fetching agent is executed with a user_id. If successful, the data moves to the next step. If it fails, the workflow stops early.

Second, process data. The processing agent extracts meaningful information from the fetched data.

Key Concept: Data Flow Between Agents

The most important idea here is data passing.

Fetcher → Processor

The output of one agent becomes the input of the next. This chaining is what makes agentic workflows powerful and scalable.

Error Handling

A simple but effective error-handling mechanism is included:

• If fetching fails, stop the workflow

• Prevent unnecessary processing

• Return a clear status message

This pattern is essential in real-world systems.

Project Structure

To run this example, organize your project like this:

your_project_directory/
├── workflow_agents/
│   ├── __init__.py
│   └── agent_definitions.py
└── main_workflow.py

Then execute:

python main_workflow.py

Example Runs

Successful case
Input: user_id = "123"
Output: Processed user information (name, occupation, user_id)

Failure case
Input: user_id = "456"
Output: Error message (“User not found”)

Why This Pattern Matters

This simple example demonstrates powerful design principles:

• Modularity: each agent does one thing well

• Reusability: agents can be reused in different workflows

• Scalability: you can easily add more agents

• Maintainability: clear separation of logic

Agent-based workflows are a foundational concept in modern AI systems and distributed architectures. Even this minimal example shows how you can break down complex tasks, build flexible pipelines, and create systems that are easy to extend.

Agent Library/Module (e.g., agent_definitions.py****)

This file will house the class definitions for our various types of

agents. These classes act as blueprints.

# workflow_agents/agent_definitions.py

class Agent:

    """

    A base class for our agents.

    While not strictly necessary for simple cases,

    it can be useful for defining common interfaces or utilities.

    """

    def __init__(self, name):

        self.name = name

        print(f"Agent '{self.name}' initialized.")

    def execute(self, data=None):

        """

        A generic method to execute the agent's task.

        Specific agents will override this.

        """

        raise NotImplementedError("Each agent must implement the

'execute' method.")

class DataFetchingAgent(Agent):

    """

    An agent specialized in fetching data.

    For this example, it will simulate fetching user data.

    """

    def __init__(self, name, data_source):

        super().__init__(name)

        self.data_source = data_source

        print(f"DataFetchingAgent will fetch from: {self.data_source}")

    def execute(self, user_id):

        """

        Simulates fetching data for a given user_id.

        In a real scenario, this might involve an API call or database query.

        """

        print(f"'{self.name}' is fetching data for user_id: {user_id}

from {self.data_source}...")

        # Simulate data fetching

        if user_id == "123":

            return {"user_id": "123", "name": "Alice Wonderland",

"occupation": "Dreamer"}

        else:

            return {"user_id": user_id, "error": "User not found"}

class DataProcessingAgent(Agent):

    """

    An agent specialized in processing data.

    For this example, it will extract key information from the fetched data.

    """

    def __init__(self, name, fields_to_extract=None):

        super().__init__(name)

        self.fields_to_extract = fields_to_extract if

fields_to_extract else ["name", "occupation"]

        print(f"DataProcessingAgent will extract fields:

{self.fields_to_extract}")

    def execute(self, fetched_data):

        """

        Processes the fetched data to extract specified fields.

        """

        print(f"'{self.name}' is processing data: {fetched_data}")

        if "error" in fetched_data:

            return {"processed_info": None, "error": fetched_data["error"]}

        processed_info = {}

        for field in self.fields_to_extract:

            processed_info[field] = fetched_data.get(field, "N/A")

        return {"processed_info": processed_info,

"original_data_keys": list(fetched_data.keys())}

agent_definitions.py:

We have a base Agent class with an __init__ method to give each agent

a name and an execute method that specific agents must implement.

DataFetchingAgent inherits from Agent. Its execute method simulates

fetching data for a user.

DataProcessingAgent also inherits from Agent. Its execute method takes

the data fetched by the previous agent and extracts specific fields.

These classes serve as the blueprints defining the agents' attributes

and methods.

Agent Instantiation and Workflow Implementation Logic (e.g., main_workflow.py)

This script will import the agent classes, create specific instances

(objects) of them, and then define the actual workflow by coordinating

these instances.

# main_workflow.py

# Import agent classes from our library

from workflow_agents.agent_definitions import DataFetchingAgent,

DataProcessingAgent

def run_user_data_workflow(user_id_to_process):

    """

    This function implements the agentic workflow.

    """

    print(f"\n--- Starting User Data Workflow for User ID:

{user_id_to_process} ---")

    # === 2. Agent Instantiation Logic ===

    # Create specific instances (objects) of agent classes and configure them.

    # Each agent can be customized for its specific role or task.

    fetcher = DataFetchingAgent(name="UserProfileFetcher",

data_source="MainUserDatabase")

    processor = DataProcessingAgent(name="UserInfoExtractor",

fields_to_extract=["name", "occupation", "user_id"])

    print("--- Agents Instantiated ---")

    # === 3. Workflow Implementation Logic ===

    # Define the sequence of tasks and how data is passed between agents.

    # Step 1: Fetch user data

    print("\nStep 1: Fetching Data...")

    fetched_user_data = fetcher.execute(user_id=user_id_to_process)

    print(f"Fetcher Output: {fetched_user_data}")

    if "error" in fetched_user_data and fetched_user_data["error"]:

        print(f"Workflow stopped due to error in fetching:

{fetched_user_data['error']}")

        return {"status": "Error", "details": fetched_user_data['error']}

    # Step 2: Process fetched data

    print("\nStep 2: Processing Data...")

    processed_data_report = processor.execute(fetched_data=fetched_user_data)

    print(f"Processor Output: {processed_data_report}")

    print("--- Workflow Completed ---")

    return {"status": "Success", "data": processed_data_report}

if __name__ == "__main__":

    # Example 1: Successful run

    result1 = run_user_data_workflow("123")

    print(f"\nFinal Result for User 123: {result1}\n")

    # Example 2: User not found

    result2 = run_user_data_workflow("456")

    print(f"Final Result for User 456: {result2}")

main_workflow.py:

Import: It first imports the agent classes defined in

workflow_agents.agent_definitions.

Agent Instantiation Logic:

Inside run_user_data_workflow, we create instances: Workspaceer =

DataFetchingAgent(...) and processor = DataProcessingAgent(...).

Each instance is configured upon creation (e.g., data_source for the

fetcher, fields_to_extract for the processor). This is where each

agent could be customized to its specific role or task.

Workflow Implementation Logic:

The workflow is defined by the sequence of calls:

Workspaceer.execute() is called first.

Its output, Workspaceed_user_data, is then passed as input to

processor.execute().

This demonstrates defining the sequence of tasks and how data is

passed between the instantiated agents.

Basic error handling is included to show how a workflow might react if

a step fails.

The if __name__ == "__main__": block shows how to trigger the workflow

with different inputs.

To make this run, you would structure your files like this:

your_project_directory/

├── workflow_agents/

│   ├── __init__.py  # This can be an empty file

│   └── agent_definitions.py

└── main_workflow.py

What Is a Ground Truth Dataset and Why It Matters in AI

In the world of artificial intelligence and machine learning, data is everything—but not all data is created equal. One of the most critical concepts behind accurate and reliable models is the ground truth dataset. Whether you're building a computer vision system, training a chatbot, or developing predictive analytics, understanding ground truth is essential.

What Is a Ground Truth Dataset?

A ground truth dataset is a collection of data that has been manually labeled or verified to represent the true, correct answer. It serves as the benchmark against which machine learning models are trained and evaluated.

For example:

• In image recognition, ground truth data might include images labeled as “cat,” “dog,” or “car.”

• In natural language processing, it could be text annotated with sentiment (positive, negative, neutral).

• In autonomous driving, it might involve annotated objects like pedestrians, traffic signs, and vehicles.

In short, ground truth = the reality your model is trying to learn.

Why Ground Truth Is So Important

Ground truth datasets are the foundation of any supervised learning system. Here’s why they matter:

1. Model Training

Machine learning models learn patterns by comparing their predictions to the correct answers provided by ground truth data. Without accurate labels, the model cannot learn effectively.

2. Evaluation and Benchmarking

Ground truth allows you to measure how well your model performs. Metrics like accuracy, precision, and recall depend entirely on comparing predictions to true labels.

3. Reducing Bias and Errors

High-quality ground truth data helps reduce bias and inconsistencies. Poor labeling leads to poor models—it’s that simple.

How Ground Truth Data Is Created

Creating a ground truth dataset is often the most time-consuming part of an AI project. It usually involves:

• Manual annotation by human experts or crowdworkers

• Quality checks to ensure consistency and accuracy

• Iterative refinement as edge cases and ambiguities are discovered

Tools like labeling platforms and annotation software are commonly used to streamline this process.

Challenges in Building Ground Truth Datasets

Despite its importance, creating ground truth data comes with challenges:

• High cost: Manual labeling can be expensive and time-intensive

• Human error: Annotators may disagree or make mistakes

• Scalability issues: Large datasets require significant resources

• Ambiguity: Some data points are subjective or unclear

Because of these challenges, many teams invest heavily in data quality assurance processes.

Best Practices

To build a reliable ground truth dataset, consider the following:

• Define clear labeling guidelines

• Use multiple annotators and measure agreement

• Regularly audit and clean your data

• Start small, then scale تدريجيًا (gradually)

• Continuously update the dataset as new data emerges

A ground truth dataset is more than just labeled data—it is the foundation of trust in any AI system. Even the most advanced algorithms cannot compensate for poor-quality ground truth. If you want better models, start with better data.

Investing time and resources into building accurate, consistent, and well-documented ground truth datasets will pay off in the long run with more reliable and robust AI systems.

Exploring the Hugging Face Ecosystem and the Diffusers Library

Hugging Face is a vibrant ecosystem made up of various libraries and websites that cater to machine learning enthusiasts and developers alike. On the library side, there are popular tools like Transformers for large language models and Diffusers, which we’ll focus on in this article. On the website side, Hugging Face provides Models, Datasets, and Spaces—all incredibly useful resources for exploring and deploying AI models.

Hugging Face Models

The Hugging Face Models hub hosts over 14,000 models, covering a wide variety of applications. You can filter these models based on your needs. For instance, if you’re interested in unconditional image generation, you can select models compatible with the Diffusers library. These models generate images without requiring a textual prompt—they simply create new visuals based on the data they were trained on.

Datasets

Hugging Face also provides a rich collection of datasets that you can use to train or fine-tune your own models. These datasets cover a wide range of domains and are fully integrated with the libraries, making it easy to experiment and build AI solutions.

Spaces

Spaces are one of the most exciting parts of Hugging Face. They allow you to try out models instantly without any coding. Most demos feature a simple Gradio interface, letting you test models interactively. Spaces are perfect for exploring new techniques and seeing the latest AI research in action.

Using the Diffusers Library for Unconditional Image Generation

Now, let’s dive into the Diffusers library and see how to generate images. Unconditional image generation means the model creates an image based solely on its training data, without any prompt or description.

Here’s a quick overview of the process:

1. Set the random seed to make your results reproducible (optional, but useful).

2. Choose a model from the Hugging Face model hub that supports unconditional image generation. For example, there are models trained on celebrity face datasets.

3. Load the diffusion pipeline using from_pretrained(model_name). This automatically downloads the model and sets up the pipeline.

4. Move the model to GPU for faster inference.

5. Generate an image by calling the pipeline. You can pass a random number generator to ensure reproducibility.

Once executed, the pipeline produces your first generated image—like a new, realistic-looking celebrity face.

Diffusion Models Explained Line by Line

1. Setup and Noise Schedule

n_steps = 512  # Total number of diffusion steps (T)
beta = linspace(start, end, n_steps)

• n_steps (T): total number of steps in the diffusion process

• beta: controls how much noise is added at each step

This is a linear noise schedule

Derived Variables

alpha = 1. - beta
alpha_bar = cumprod(alpha, axis=0)

• alpha: amount of signal preserved at each step

• alpha_bar: cumulative product → how much of the original image remains after t steps

sqrt_alpha_bar = sqrt(alpha_bar)
sqrt_one_minus_alpha_bar = sqrt(1. - alpha_bar)

These are used in the reparameterization trick:

[
x_t = \sqrt{\bar{\alpha}_t} x_0 + \sqrt{1 - \bar{\alpha}_t} \epsilon
]

Model and Optimizer

model = UNet()
optimizer = Adam(model.parameters(), lr=0.001)

• UNet: predicts the noise given a noisy image and timestep

• Adam: optimizer used to train the model

TRAINING (Forward Process)

for batch, _ in dataloader:

Loop over real images.

Batch size

bs = batch.shape[0]

Sampling random timesteps

t = torch.randint(0, T, (bs,)).long()

Very important:

• Each image uses a different timestep

• The model learns to handle all noise levels

Generate noise

noise = torch.randn_like(batch, device=device)

• Gaussian noise (ε)

Add noise to images

x_noisy = (
    sqrt_alpha_bar[t].view(bs, 1, 1, 1) * batch +
    sqrt_one_minus_alpha_bar[t].view(bs, 1, 1, 1) * noise
)

This is the forward diffusion step.

• Early timesteps → image is mostly clean

• Late timesteps → image becomes almost pure noise

Predict the noise

noise_pred = model(x_noisy, t)

The model learns:

“Given a noisy image and timestep, what noise was added?”

Loss function

loss = F.mse_loss(noise, noise_pred)

We minimize:

[
||\epsilon - \epsilon_\theta||^2
]

Optimization

loss.backward()
optimizer.step()

Update model weights.

3. IMAGE GENERATION (Reverse Process) Inference 

Now we generate new images from pure noise.

Precomputations

sqrt_one_minus_alpha_bar = sqrt(1. - alpha_bar)

alpha_bar_t_minus_1 = F.pad(alpha_bar[:-1], (1, 0), value=1.0)

• Shifted version of alpha_bar

• Needed for reverse formulas

posterior_variance = (
    beta * (1.0 - alpha_bar_t_minus_1) / (1.0 - alpha_bar)
)

This corresponds to:

[
\sigma_t^2
]

Controls how much randomness is added during generation

Initialization

bs = 8
x = randn((bs, 3, IMG_SIZE, IMG_SIZE))

Start from pure noise.

Reverse loop

for ts in range(0, T)[::-1]:

Go backward in time (T → 0)

Add noise (except final step)

noise = randn_like(x) if ts > 0 else 0

Important detail:

• No noise at the final step

• Otherwise the image would degrade

Time tensor

t = full((bs,), ts).long()

Denoising step

x = (
    sqrt_one_over_alpha[t].view(bs, 1, 1, 1) *
    (
        x - beta[t].view(bs, 1, 1, 1) /
        sqrt_one_minus_alpha_bar[t].view(bs, 1, 1, 1) *
        model(x, t)
    )
    + sqrt(posterior_variance[t].view(bs, 1, 1, 1)) * noise
)

What happens here?

Each step:

1. The model predicts the noise

2. That noise is removed

3. A small amount of controlled noise is added back

Missing definition (important)

sqrt_one_over_alpha = sqrt(1.0 / alpha)

Final Output

generated_image = torch.clamp(x, -1, 1)

Clamp values to a valid image range.

Intuition Recap

Training

• Add noise to images

• Train model to predict that noise

Generation

• Start from noise

• Remove noise step by step

• Obtain a clean 

From Mixed to Pure Understanding Qubit Purification and Decoherence

Quantum computing introduces a radically different way of thinking about information. At its core lies the qubit, the quantum analogue of the classical bit. Unlike classical bits, which are strictly 0 or 1, qubits can exist in superpositions of both states. However, this powerful feature comes with a fundamental challenge: qubits are extremely fragile.

In this article, we explore the concept of purification in quantum mechanics—what it means, why it matters, and how it connects to one of the biggest obstacles in quantum computing: decoherence.

Pure vs. Mixed States

To understand purification, we first need to distinguish between two types of quantum states.

Pure states are fully known quantum states. A qubit in a pure state can be described precisely by a wavefunction:

ψ = α|0⟩ + β|1⟩

Here, α and β are complex numbers that encode probabilities and phase information. This is the ideal situation in quantum computing, where the system is perfectly controlled.

Mixed states, on the other hand, represent uncertainty. They arise when we don’t have complete information about the system. For example, if a qubit has a 50% chance of being |0⟩ and 50% chance of being |1⟩—but not in superposition—then it is in a mixed state. This is not quantum uncertainty, but classical ignorance.

What Is Purification?

Purification is a powerful idea.

Every mixed state can be seen as part of a larger system that is actually in a pure state.

In other words, a qubit that appears noisy or impure may not be fundamentally disordered—it may simply be entangled with something else.

By expanding our view to include an additional system, often called an ancilla or external qubit, we can represent the combined system as a pure state.

A Simple Example

Consider a system of two qubits in this entangled state:

Ψ = (1/√2)(|00⟩ + |11⟩)

This is a perfectly pure state. However, if we look at only one of the two qubits and ignore the other, the remaining qubit appears to be in a mixed state: 50% |0⟩ and 50% |1⟩.

What happened?

The missing information is not gone. It is stored in the correlation between the two qubits. This is the essence of purification: the apparent randomness comes from ignoring part of a larger, structured system.

Decoherence The Real-World Challenge

In practice, qubits are never perfectly isolated. They constantly interact with their environment—vibrations, electromagnetic fields, temperature fluctuations, and more.

These interactions cause decoherence, a process where the qubit becomes entangled with its environment, quantum superposition is lost, and the system begins to behave like a classical mixture.

From the perspective of purification, the environment acts as the external system that carries away information. The total system, qubit plus environment, is still in a pure state, but the qubit alone appears mixed and degraded.

Why Purification Matters

Understanding purification is not just a theoretical exercise. It has practical implications.

Quantum error correction relies on tracking and managing entanglement with auxiliary systems. Quantum communication protocols use purification to restore high-quality entanglement. Quantum cryptography depends on distinguishing pure entangled states from mixed ones.

In essence, purification helps us reinterpret noise not as destruction of information, but as redistribution of it.

Key Insight

A mixed state is not necessarily a sign of disorder. It is often a sign that part of the system is hidden.

By identifying and including the external degrees of freedom, what seemed random becomes structured again.

Quantum systems challenge our classical intuition. What appears as noise may actually be hidden order. What seems like loss may be entanglement. Purification provides a lens through which we can better understand these phenomena and learn how to control them.

As quantum technologies continue to evolve, mastering concepts like purification and decoherence will be essential for building reliable and scalable quantum systems.

Text Generation with PyTorch: Character-Level vs Subword Tokenization

Artificial Intelligence has made enormous progress in Natural Language Processing (NLP), especially in the area of text generation. Modern systems can generate realistic text by learning patterns from large datasets.

In this tutorial, we will build a text generation model using PyTorch trained on a dataset containing works from Shakespeare. We will explore two different approaches:

1. Character-level text generation

2. Subword tokenization using Hugging Face

By the end of this article, you will understand how language models generate text token by token.

Dataset: Shakespeare Text

To train the model, we use a dataset containing Shakespeare's writings.

DATA_FILE = '../data/shakespeare_small.txt'

with open(DATA_FILE, 'r') as data_file:
    raw_text = data_file.read()

print(f'Number of characters in text file: {len(raw_text):,}')

This dataset contains thousands of characters that will be used to train our neural network to predict the next token in a sequence.

Character-Based Text Generation

In the first approach, each character is treated as a token.

Example:

"hello" → ['h', 'e', 'l', 'l', 'o']

The model learns to predict the next character based on the previous ones.

Step 1 — Text Normalization

Before tokenization, we normalize the text.

def normalize_text(text: str) -> str:
    normalized_text = text.lower()
    return normalized_text

This converts all text to lowercase while keeping punctuation and special characters.

Step 2 — Pretokenization

Next, we split the text into characters.

def pretokenize_text(text: str) -> list[str]:
    smaller_pieces = [char for char in text]
    return smaller_pieces

Step 3 — Tokenization

Now we combine normalization and pretokenization.

def tokenize_text(text: str) -> list[str]:
    normalized_text = normalize_text(text)
    pretokenized_text = pretokenize_text(normalized_text)
    tokenized_text = pretokenized_text
    return tokenized_text

Step 4 — Encoding Tokens into IDs

We convert tokens into integer IDs.

encoded_text, character_mapping = encode_text(raw_text, tokenize_text)

The mapping allows conversion between characters and integer IDs.

character → integer ID
integer ID → character

Preparing the Dataset

Next, we split the encoded text into sequences.

sequence_length = 32
batch_size = 32

train_dataset = ShakespeareDataset(encoded_text, sequence_length)

train_loader = DataLoader(
    train_dataset,
    shuffle=False,
    batch_size=batch_size,
)

Each sequence contains 32 characters, and the model learns to predict the next character.

Building the Model

We create a neural network using a helper function.

model = build_model(n_tokens)
model.to(device)

criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters())

CrossEntropyLoss is used for classification tasks, while the Adam optimizer updates model weights during training.

Text Generation Function (Character-Based)

The following function generates text character by character.

def generate_text_by_char(
    input_str: str,
    model,
    token_mapping,
    num_chars: int = 100,
    temperature: float = 1.0,
    topk: int | None = None,
):

    tokenized_text = tokenize_text(input_str)
    generated_tokens = []

    for _ in range(num_chars):

        new_char = next_token(
            tokenized_text=(tokenized_text + generated_tokens),
            model=model,
            token_mapping=token_mapping,
            temperature=temperature,
            topk=topk,
            device=device,
        )

        generated_tokens.append(new_char)

    full_text = ''.join(tokenized_text + generated_tokens)
    return full_text

Training the Model

Now we train the model.

TEST_PHRASE = 'To be or not to be'
epochs = 5 if device == 'cpu' else 25

start = start_time()

for epoch in range(epochs):

    model.train()
    total_loss = 0

    for X_batch, y_batch in train_loader:

        optimizer.zero_grad()

        output = model(X_batch.to(device))

        loss = criterion(output.transpose(1, 2), y_batch.to(device))

        loss.backward()

        optimizer.step()

        total_loss += loss.item()

    print(f'Epoch {epoch + 1}/{epochs}, Loss: {total_loss / len(train_loader)}')

    gen_output = generate_text_by_char(
        input_str=TEST_PHRASE,
        model=model,
        num_chars=100,
    )

    print(gen_output)

During training, we generate text after each epoch to observe improvements.

Generating Text

Once the model is trained, we can generate new text.

output = generate_text_by_char(
    input_str='To be or not to be',
    model=model,
    num_chars=100,
    temperature=1.0,
    topk=None,
)

print(output)

Adjusting parameters changes creativity.

Temperature controls randomness, while top-k sampling restricts token selection.

Subword-Based Text Generation

Character-based models are simple but inefficient. Modern NLP models use subword tokenization.

We use a tokenizer from the Transformers library.

Loading a Hugging Face Tokenizer

from transformers import AutoTokenizer

model_name = 'bert-base-uncased'

my_tokenizer = AutoTokenizer.from_pretrained(
    model_name,
)

This tokenizer splits words into subword units.

Example:

playing → play + ##ing

Encoding Text with the Tokenizer

encoded_text, token_mapping = encode_text_from_tokenizer(
    text=raw_text,
    tokenizer=my_tokenizer,
)

Each token is converted into an integer ID.

Preparing the Dataset

sequence_length = 32
batch_size = 32

train_dataset = ShakespeareDataset(encoded_text, sequence_length)

train_loader = DataLoader(
    train_dataset,
    shuffle=False,
    batch_size=batch_size,
)

Subword Text Generation Function

def generate_text_by_subword(
    input_str: str,
    model,
    token_mapping,
    tokenizer,
    num_tokens: int = 100,
    temperature: float = 1.0,
    topk: int | None = None,
):

    tokenized_text = tokenize_text_from_tokenizer(
        tokenizer=tokenizer,
        text=input_str,
    )

    generated_tokens = []

    for _ in range(num_tokens):

        new_token = next_token(
            tokenized_text=(tokenized_text + generated_tokens),
            model=model,
            token_mapping=token_mapping,
            temperature=temperature,
            topk=topk,
            device=device,
        )

        generated_tokens.append(new_token)

    output_ids = tokenizer.convert_tokens_to_ids(
        tokenized_text + generated_tokens
    )

    full_text = tokenizer.decode(output_ids)

    return full_text

Generating Subword Text

output = generate_text_by_subword(
    input_str='To be or not to be',
    model=model,
    token_mapping=token_mapping,
    tokenizer=my_tokenizer,
    num_tokens=30,
    temperature=1.5,
    topk=100,
)

print(output)

This approach generally produces more coherent text than character-level models.

Character vs Subword Models

Character-based models are simple and easy to implement but slower and less efficient.

Subword-based models are more powerful and scalable but require a tokenizer.

Most modern NLP models use subword tokenization because it balances vocabulary size and language coverage.

How to Create an Agent Ideation Partner for Copilot Studio

Purpose of the Agent Ideation Partner

This agent does not execute operational tasks.
It does not directly automate processes.
It does not run real-time analytics.

Instead, it performs one strategic function: it analyzes the user’s last 90 days of internal work context and generates tailored agent ideas that improve productivity.

How the Agent Works

The Agent Ideation Partner is designed to review internal data from the past 90 days, including emails, meeting transcripts, chat messages, and internal documents.

It identifies core responsibilities, recurring tasks, bottlenecks, frequently asked internal questions, and low-value manual activities.

Based on this understanding, it generates lightweight agent ideas compatible with Copilot Studio, evaluates user-proposed ideas for feasibility, and provides expert-level recommendations focused specifically on “lite” agents.

Example Prompt to Create the Agent

Prompt: Agent Ideation Partner

Create an Agent Ideation Partner focused on generating agent use cases specifically for the user’s role within their company.

This agent must analyze internal data from the last 90 days (emails, meeting transcripts, chats, internal documents) and understand primary responsibilities, recurring activities, frequent challenges, and key collaborations. It must identify repetitive patterns and inefficiencies.

Based on this analysis, the agent must suggest personalized agent use cases and propose only ideas suitable for Copilot Studio Agent Builder. It should prioritize lightweight automations, FAQ bots, smart reminders, automated draft generation, and meeting preparation assistants.

For each idea, it must explain the problem it solves, how the agent would work, what data it would require, and why it qualifies as a “lite” implementation.

If the user already has agent ideas, the agent must evaluate their feasibility in Copilot Studio, recommend simplifications, and highlight limitations or better alternatives.

The agent must not propose complex enterprise systems, custom software architectures, or advanced external integrations beyond standard connectors. The focus must remain exclusively on lightweight, quick-to-implement, high-value productivity agents.

Example Use Cases This Agent Might Generate

Meeting Preparation Agent
Problem: Preparing for meetings requires manually gathering emails, documents, and open action items.
Solution: An agent that automatically generates a meeting briefing.
Required data: Recent emails, calendar events, shared documents.
Why it’s lightweight: Simple aggregation and summarization logic.

Internal Team FAQ Agent
Problem: The same operational questions are repeatedly asked in chat.
Solution: An agent that answers common process, template, and policy questions.
Required data: Internal documents and knowledge base materials.
Why it’s lightweight: Structured conversation flows using existing documentation.

Task Follow-Up Assistant
Problem: Action items from meetings are not consistently tracked.
Solution: An agent that extracts action items and sends periodic reminders.
Required data: Meeting transcripts and calendar data.
Why it’s lightweight: Basic conditional logic and scheduled notifications.

Weekly Report Draft Generator
Problem: Weekly reports are repetitive and time-consuming.
Solution: An agent that aggregates recent activity and generates a structured draft.
Required data: Sent emails, completed tasks, updated documents.
Why it’s lightweight: Template-based generation with summarized inputs.

Why This Approach Works

Many teams aim to build transformational AI agents immediately. That often leads to overly complex initiatives.

The fastest return on investment comes from micro-automations, removing daily friction, reducing repetitive manual work, and supporting decision-making with contextual summaries.

A well-designed Agent Ideation Partner helps organizations avoid overly ambitious projects, identify quick wins, align AI initiatives with real job functions, and build a phased roadmap of practical agents.

You don’t need complex enterprise AI to create impact. You need context-aware, role-specific, lightweight agents.

An Agent Ideation Partner built with a structured prompt like this transforms internal work patterns into actionable automation opportunities and enables fast, measurable value using Copilot Studio.

Create an Agent Ideation Partner focused on generating agent use
cases specifically for the user's role at my company. This agent
should review all internal data from the 90 days —including emails,
meeting transcripts, chat messages, and internal documents—to
understand the user's responsibilities, challenges, and recurring
tasks. Based on this context, suggest personalized agent use cases
that would improve user's productivity, automate repetitive tasks,
or enhance collaboration. Prioritize ideas that align with user's
role and current projects and explain how each agent would work and
what data it would need.
This agent should ONLY create ideas for Copilot Studio Agent
Builder. Focus on ideas that are lightweight and simple to implement
and solve common productivity or FAQ scenarios.
If the users already have some agent ideas to build, this agent
should advise the users if the use case would be a good fit for
Copilot Studio Agent Builder and give recommendations and
suggestions with deep expertise in lite agents.

To make Generative Adversarial Networks (GANs) work well in practice

To make Generative Adversarial Networks (GANs) work well in practice, choosing the right architecture is crucial.

If you're working on a simple task—like generating 28×28 pixel handwritten digits from the MNIST dataset—you can often use a fully connected architecture. In this setup, each layer interacts through matrix multiplications only. There’s no convolution, no recurrence—just dense layers stacked together.

The key design rule? Both the generator and the discriminator must include at least one hidden layer. This ensures they have the universal approximator property, meaning that, given enough hidden units, they can represent any probability distribution.

Choosing Activation Functions

For hidden layers, many activation functions can work. However, Leaky ReLU is especially popular. The reason is simple: it helps gradients flow through the network more reliably.

Gradient flow is important in any neural network—but it’s absolutely critical in GANs. Why? Because the generator can only learn through the gradients it receives from the discriminator. If gradients vanish, learning stalls.

For the generator’s output layer, a common choice is the hyperbolic tangent (tanh) activation function. This means your training data should be scaled to the range [-1, 1].

For the discriminator, the output must represent a probability. To enforce this, we typically use a sigmoid activation function at the output layer.

Training GANs: Two Optimizers, Two Losses

GANs differ from most machine learning models because they require training two networks simultaneously:

• One optimizer minimizes the discriminator’s loss.

• Another optimizer minimizes the generator’s loss.

A widely used optimizer is Adam, a design choice popularized in the DCGAN architecture.

Designing the Discriminator Loss

The discriminator’s goal is straightforward:

• Output values close to 1 for real data.

• Output values close to 0 for fake (generated) data.

This is essentially a binary classification problem. Therefore, the correct loss function is sigmoid cross-entropy, just like in standard classifiers.

One common mistake is implementing cross-entropy incorrectly. You should always use the numerically stable version computed directly from the logits (the values before the sigmoid). Using probabilities after the sigmoid can cause numerical instability, especially when outputs are very close to 0 or 1.

Label Smoothing Trick

A GAN-specific trick is to slightly soften the real labels. Instead of labeling real examples as 1, use something like 0.9. Keep fake labels at 0.

This technique—called label smoothing—prevents the discriminator from becoming overly confident and improves generalization.

Designing the Generator Loss

For the generator, we also use cross-entropy—but with flipped labels. In other words, the generator tries to make the discriminator classify fake data as real.

Some implementations use the negative of the discriminator’s loss as the generator’s loss. While intuitive, this approach doesn’t work well in practice. When the discriminator is performing well, its gradients can vanish—leaving the generator with no meaningful signal to learn from.

Instead, the better approach is to let the generator minimize cross-entropy with flipped labels. This works because:

• The derivative of cross-entropy remains non-zero unless the loss is fully minimized.

• The losing player always receives gradient feedback.

• Training remains stable and effective.

Final Thoughts

To summarize best practices for simple GAN architectures:

• Use fully connected layers for simple datasets like MNIST.

• Ensure both networks have at least one hidden layer.

• Prefer Leaky ReLU for hidden activations.

• Use tanh for the generator output (and scale data to [-1, 1]).

• Use sigmoid for the discriminator output.

• Apply numerically stable cross-entropy computed from logits.

• Consider label smoothing for real samples.

• Train generator and discriminator with separate optimizers (Adam is a strong choice).

• Let both networks minimize cross-entropy—don’t rely on maximizing the discriminator’s loss.

Getting these architectural and optimization details right can make the difference between a GAN that fails to train and one that produces convincing results.

GANs Equilibriaerative

To understand GANs, we need to think about payoffs and equilibria in the context of machine learning. If we can identify an equilibrium in a GAN game, we can use that equilibrium as a defining characteristic to better understand how the game works.

Most machine learning models we have seen so far are based on optimization. We define some model parameters, write down a cost function of those parameters, and then minimize that cost. If we imagine plotting this, the parameters lie along the horizontal axes and the cost function is shown on the vertical axis. The goal of the learning algorithm is to find a very low value of the cost function. In practice, we usually do not reach a global or even a local minimum. Instead, we often get stuck in regions where numerical issues make further progress difficult, but the optimization algorithm still manages to find a point with reasonably low cost.

GANs are different because there are two players: the generator and the discriminator. Each has its own cost function. In a simple and easy-to-visualize version of GANs, the discriminator’s cost is just the negative of the generator’s cost. In this case, we can describe the entire interaction using a single value function. The generator tries to minimize this value function, while the discriminator tries to maximize it.

Recall that an equilibrium occurs when neither player can improve their outcome without the other changing their strategy. In this setting, that happens at a point that is a maximum for the discriminator and a minimum for the generator. Such a point is called a saddle point. If we imagine a graph of the game, the equilibrium lies in the middle.

One horizontal axis represents the generator’s parameters. If we take a cross-section along this axis, the equilibrium appears as a minimum of the value function. The other horizontal axis represents the discriminator’s parameters. Along that axis, the equilibrium appears as a maximum of the value function.

This perspective helps us understand games played by deep networks. We aim to find a point in the joint parameter space that is simultaneously a local minimum for each player’s cost with respect to that player’s own parameters.

When we analyze the GAN game, the discriminator reaches a local maximum when it correctly estimates the probability that an input is real rather than fake. This probability is given by the ratio between the data density at that input and the sum of the data density and the generator’s model density at that input. Intuitively, this ratio measures how much of the probability mass in a region comes from real data rather than from the generator.

Using tools from game theory, we can show that if both networks are sufficiently expressive, there exists an equilibrium where the generator’s density matches the true data density. At that point, the discriminator outputs a constant value everywhere (for example, 1/2 in the standard formulation).

However, even though such an equilibrium exists, finding it in practice is difficult. We usually train GANs by running two optimization algorithms simultaneously, each minimizing its own player’s cost with respect to that player’s parameters. Unfortunately, these optimization procedures do not necessarily converge to the game’s equilibrium.

A common failure case occurs when the data distribution has multiple clusters. The generator may learn to produce samples from only one cluster. The discriminator then learns to reject that cluster as fake, prompting the generator to switch to a different cluster, and so on. Instead of covering all clusters at once, the generator keeps cycling between them.

Ideally, we would like a training algorithm that reliably finds the equilibrium where the generator captures all clusters simultaneously. Designing such an algorithm for high-dimensional, continuous, non-convex games remains one of the key research challenges in GANs. Solving this problem could unlock new applications that depend on highly reliable learning algorithms.

Building a Cognitive Flow Engine: Moving Beyond Simple Prompting

The Limitation of Just Prompting

With a standard prompt, the interaction looks like this:

• You ask a question.

• The model responds.

• You refine.

• It answers again.

Each response is shaped by the latest message plus conversation memory, but there is no structured evolution of thinking. No designed arc. No cognitive progression. It works, but it feels flat. There’s no deliberate shift between exploration, connection, synthesis, and insight. That’s where a state-based engine makes the difference.

From Prompting to Cognitive States

Instead of treating every message equally, I introduced cognitive states:

• Exploration → Generate perspectives and possibilities

• Connection → Link ideas in unexpected but coherent ways

• Synthesis → Distill and organize

• Insight → Translate into action and reflection

Each state subtly changes the tone, structure, depth, and level of abstraction. The model doesn’t change its intelligence — but it changes its mode of reasoning. This small shift produces a surprisingly big experiential difference.

Why Not Just Write It in the Prompt?

Technically, you could write:

"Respond in four phases: explore, connect, synthesize, and provide insights."

And it would work. But a single prompt gives you a formatted output. A state engine gives you a designed thinking journey. With an engine:

• The state persists.

• The tone adapts automatically.

• The interaction becomes modular.

• You can switch modes intentionally.

It becomes programmable cognition instead of formatted text generation. The engine acts as a cognitive conductor guiding the rhythm of reasoning.

How the Cognitive Flow Engine Works

The structure is simple:

1. Define cognitive states.

2. Assign each state a system-level instruction.

3. Inject the appropriate instruction before each response.

4. Optionally adjust model parameters (temperature, top_p).

5. Allow manual or automatic state transitions.

No gimmicks. No artificial limits. Just controlled modulation of reasoning style.

Practical Applications

This approach is powerful for:

Creative Work: Brainstorming ideas, naming products, developing campaigns.
Strategic Thinking: Exploring options first, then synthesizing into a roadmap.
Writing: Generating raw material in exploration mode, then polishing in synthesis mode.
Decision-Making: Mapping perspectives, finding connections, extracting actionable next steps.

The Real Insight

The real breakthrough isn’t about creativity. It’s about structured cognitive progression. Most AI interactions are reactive. Human thinking is phased: we explore, connect, refine, and act. When your AI system mirrors that structure, it feels less like a tool and more like a thinking partner.

Where This Is Going

State-based engines can evolve into focus modes, contrarian modes, strategic compression modes, deep research modes, or reflective coaching modes. The future of AI interaction isn’t just better prompts. It’s programmable cognitive architecture. Once you start designing thinking states instead of writing single prompts, you won’t want to go back.

Example Code

Here’s a minimal implementation in JavaScript:

export const COGNITIVE_STATES = {
  EXPLORATION: "exploration",
  CONNECTION: "connection",
  SYNTHESIS: "synthesis",
  INSIGHT: "insight",
};

export default class CognitiveFlowEngine {
  constructor(agent) {
    this.agent = agent;
    this.currentState = COGNITIVE_STATES.EXPLORATION;
  }

  setState(state) {
    if (COGNITIVE_STATES[state.toUpperCase()]) {
      this.currentState = state;
    }
  }

  wrapUserText(userText) {
    return `
User request:
${userText}

Instructions:
- Respond according to the current cognitive state.
- Keep responses clear and practical.
- Add metaphor only if helpful.
`;
  }

  async respond(userText) {
    const prompt = this.wrapUserText(userText);
    return await this.agent.generate({ prompt });
  }
}

This code gives you a stateful wrapper around any LLM agent, letting you dynamically guide reasoning style without imposing limits or a strict turn system.

Wikipedia Code (Wikitext)

1. Basic Text Formatting

Bold Text

'''Bold text'''

Result: Bold text

Italic Text

''Italic text''

Result: Italic text

Bold and Italic

'''''Bold and Italic'''''

2. Headings

Headings structure your article.

= Main Title =
== Section ==
=== Subsection ===
==== Smaller Subsection ====

Wikipedia automatically generates a table of contents if there are multiple sections.

3. Internal Links

Internal links connect to other Wikipedia pages.

[[Italy]]

Custom text:

[[Italy|beautiful country]]

4. External Links

Link to external websites:

[https://www.example.com]

With text:

[https://www.example.com Official Website]

5. Lists

Bullet List

* First item
* Second item
* Third item

Numbered List

# First
# Second
# Third

Sub-lists

* Item
** Sub-item

6. References (Citations)

Citations are mandatory for reliability.

<ref>Author, Title, Website, Date</ref>

At the bottom of the page:

== References ==
{{Reflist}}

7. Citation Templates

Professional citation format:

<ref>{{Cite web
 |title=Article Title
 |url=https://example.com
 |website=Website Name
 |date=2024-01-01
 |access-date=2024-02-01
}}</ref>

Other citation templates include:

• {{Cite book}}

• {{Cite news}}

• {{Cite journal}}

8. Infobox

Infoboxes appear on the right side of an article.

Example for a person:

{{Infobox person
 | name = John Doe
 | birth_date = 1990
 | occupation = Writer
}}

Each topic has its own infobox template.

9. Images

[[File:Example.jpg|thumb|Caption text]]

Options:

• thumb → thumbnail

• left / right → alignment

• 300px → size

Example:

[[File:Example.jpg|thumb|300px|Description]]

10. Categories

Categories go at the bottom of the page.

[[Category:Writers]]
[[Category:Living people]]

11. Templates

Templates reuse structured content.

Basic usage:

{{TemplateName}}

With parameters:

{{TemplateName|parameter=value}}

12. Tables

Basic table example:

{| class="wikitable"
! Name !! Age
|-
| John || 30
|-
| Anna || 25
|}

13. Comments (Hidden Text)

<!-- This is a comment -->

Visible only in edit mode.

14. Draft Submission (English Wikipedia)

To submit a draft for review:

{{subst:AFC submission/submit}}

Used only in the Draft namespace.

How CNNs Learn to “See”: Filters, Feature Maps, and Color Images

When we look at an image, even a simple photo of a dog, our brain can recognize many details at the same time: teeth, whiskers, tongue, eyes. A Convolutional Neural Network (CNN) works in a similar way, but it does so by using filters.

One Region, Many Patterns

A single region of an image can contain multiple visual features. To properly understand it, a CNN does not rely on just one filter, but on multiple filters operating in parallel, each designed to detect a different pattern such as edges, curves, or textures.

Each filter creates its own collection of nodes in the convolutional layer. These nodes share the same weights with each other, but their weights are different from those of other filters. In practice, modern CNNs often use dozens or even hundreds of filters in a single convolutional layer.

Feature Maps and Activation Maps

As each filter slides across the image, it produces a matrix of values. These matrices are known as:

• Feature maps

• Activation maps

Visually, feature maps look like filtered versions of the original image. They contain less information, but only what is relevant to the specific pattern the filter is designed to detect.

For example, if we apply four 4×4 filters to an image (such as one from a self-driving car dataset), we obtain four feature maps. Some may highlight vertical edges, while others emphasize horizontal edges.

Brighter values in a feature map indicate that the pattern defined by the filter was strongly detected in that region of the image.

Why Edge Detection Matters

Edges are one of the most important visual features in images. They often appear as a line of light pixels next to a line of darker pixels. Because of this, edge-detecting filters play a crucial role in CNNs, especially in the early layers.

By matching the bright regions in a feature map with the original image, we can see which parts of the image activated the filter.

What About Color Images?

So far, we have considered grayscale images, which computers interpret as 2D arrays (height × width).
Color images, however, are represented as 3D arrays:

• height

• width

• depth (three channels: red, green, and blue)

An RGB image can be thought of as a stack of three 2D matrices, one for each color channel.

To convolve a color image, the filter must also be three-dimensional. Each filter contains weights for the red, green, and blue channels at every spatial location. The convolution process is the same as before, but the sum now includes values from all three channels.

Multiple Filters and Increasing Depth

When we apply multiple filters to a color image, we obtain multiple feature maps. These maps can be stacked together to form a new 3D array, which then becomes the input to the next convolutional layer.

This is how CNNs gradually learn:

• patterns

• patterns within patterns

• and increasingly complex visual structures

CNNs vs. Dense Layers

In many ways, convolutional layers are similar to fully connected (dense) layers:

• both use weights and biases

• both rely on backpropagation

• both minimize a loss function

The key difference is connectivity:

• dense layers are fully connected

• convolutional layers are locally connected and use shared weights

Learning Happens Automatically

When building a CNN:

• filter weights are initialized randomly

• we do not specify which patterns to detect

• we only define a loss function (for example, categorical cross-entropy)

During training, backpropagation updates the filters to minimize the loss. As a result, the network learns on its own which patterns are useful for the task.

Are CNNs, MLPs, and Classical Neural Networks Obsolete?

A Scientific and Systems-Level Perspective on Modern Artificial Intelligence

Artificial intelligence evolves at a remarkable pace, and each new breakthrough tends to generate the same recurring narrative: that whatever came before is now obsolete. In recent years, the rise of Transformer-based architectures, Large Language Models, and multimodal foundation models has led many to question the relevance of classical neural networks such as Multi-Layer Perceptrons (MLPs) and Convolutional Neural Networks (CNNs). This perception, however, overlooks the nuanced way in which progress in machine learning actually occurs.

Classical architectures have not been replaced; rather, they have become specialized, embedded, and indispensable components of modern AI systems. Understanding their continuing relevance requires moving beyond benchmark hype and examining neural networks from scientific, architectural, and systems-engineering perspectives. The question is therefore not whether CNNs or MLPs are obsolete, but whether they are being evaluated in the correct context.

Multi-Layer Perceptrons (MLPs) represent the most basic form of neural computation. Mathematically, an MLP is a sequence of affine transformations followed by nonlinear activation functions, capable of approximating arbitrary continuous functions under mild conditions. This theoretical property, formalized by the Universal Approximation Theorem, establishes MLPs as powerful function approximators. However, expressive power alone does not guarantee efficiency or generalization. MLPs make almost no assumptions about the structure of the data, which is both their greatest strength and their primary limitation.

In domains where data is structured, low-dimensional, and already encoded in meaningful features, such as finance, healthcare records, industrial telemetry, or business analytics, this lack of inductive bias becomes an advantage. MLPs often outperform more complex architectures in these settings because they introduce less variance, require fewer samples to generalize, and are computationally efficient. For this reason, MLPs remain widely used in production systems, particularly for tabular data. Far from disappearing, they are also deeply embedded within modern architectures. Every Transformer block contains large feed-forward networks that are, in essence, MLPs. Classification heads, regression layers, reinforcement learning policies, and Mixture-of-Experts (MoE) components all rely on MLPs as fundamental building blocks. Declaring MLPs obsolete would therefore imply declaring most modern AI architectures obsolete as well.

Convolutional Neural Networks (CNNs) were introduced to address a different limitation of early neural models: the inability to exploit spatial structure. Images, videos, and other grid-like data exhibit strong local correlations and translational regularities. CNNs encode these assumptions directly through local receptive fields, weight sharing, and hierarchical feature extraction. This architectural bias allows CNNs to learn visual representations efficiently, using far fewer parameters than fully connected networks. From a biological and computational perspective, CNNs are remarkably well aligned with human visual processing. Early layers detect edges and simple patterns, intermediate layers capture textures and shapes, and deeper layers represent object-level semantics. This hierarchical organization enables CNNs to generalize well even when training data is limited. As a result, CNNs remain the backbone of countless real-world systems, including autonomous driving perception stacks, medical imaging diagnostics, satellite image analysis, industrial inspection pipelines, and mobile vision applications.

The emergence of Vision Transformers has led to renewed debate about the future of CNNs. Vision Transformers replace convolutional inductive bias with global self-attention, allowing every image patch to interact with every other patch. While this approach achieves impressive results on massive datasets, it comes at the cost of increased computational complexity and reduced data efficiency. In practice, CNNs often outperform Vision Transformers in small- and medium-scale regimes, especially when latency, energy consumption, and robustness matter. Many state-of-the-art vision systems now adopt hybrid designs, using CNNs for efficient feature extraction and Transformers for high-level reasoning. CNNs are not obsolete; they are efficient specialists integrated into larger systems.

Sequential data introduces yet another structural challenge. Language, speech, sensor streams, and time series depend on temporal order. Recurrent Neural Networks (RNNs) were designed to model this dependency by maintaining a hidden state that evolves over time. Vanilla RNNs, while conceptually elegant, suffer from optimization difficulties that limit their ability to capture long-range dependencies. Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs) addressed these issues by introducing gating mechanisms that regulate information flow, enabling stable learning over long sequences. Although Transformers have largely replaced recurrent models in large-scale natural language processing, LSTMs and GRUs remain highly relevant in domains where streaming inference, low latency, or limited data are critical. In many real-time systems, such as speech recognition on edge devices or industrial time-series monitoring, recurrent models offer a better trade-off between performance and efficiency than attention-based architectures.

Transformers themselves represent a powerful but often misunderstood paradigm. By relying on self-attention, Transformers can model global dependencies without recurrence, enabling massive parallelization and scalability. This makes them ideal for large datasets and general-purpose modeling, which is why they form the foundation of modern language and multimodal models. Beyond Transformers, the modern AI landscape has grown to include several specialized architectures and training paradigms. Large Language Models (LLMs) are Transformer-based architectures trained on massive text corpora, often extended to multimodal capabilities. Mixture-of-Experts (MoE) architectures distribute computation across specialized subnetworks, improving scalability while retaining high performance. Vision-Language Models (VLMs) integrate visual and textual inputs, combining CNNs or Vision Transformers with MLPs and attention layers to achieve multimodal reasoning. Sequence Learning Models (SLMs) continue to process temporal data efficiently, building on recurrent or Transformer-based foundations, while Masked Language Models (MLMs) employ objectives in which certain tokens are hidden and predicted, as exemplified by BERT. Segment Anything Models (SAMs) demonstrate universal image segmentation capabilities using hybrid CNN and Transformer backbones. Even less standardized terms, such as Latent Concept Models (LCMs) or Language-Aware Models (LAMs), highlight ongoing efforts to adapt classical components to novel data representations and task-specific biases.

Beyond grids and sequences lies relational data, which is best modeled as graphs. Graph Neural Networks (GNNs) introduce a message-passing framework that allows node representations to evolve based on local neighborhood structure. This inductive bias is irreplaceable for tasks involving social networks, molecular structures, recommendation systems, and knowledge graphs. No amount of attention or convolution can fully substitute for architectures explicitly designed to operate on graphs. GNNs are therefore not alternatives to CNNs or Transformers, but complementary tools tailored to relational reasoning.

Generative modeling further illustrates the cumulative nature of AI progress. Autoencoders (AEs) and Variational Autoencoders (VAEs) introduced principled approaches to representation learning and probabilistic modeling. While newer techniques such as diffusion models have achieved superior generative quality, they rely heavily on classical components. Most diffusion models use convolutional U-Net architectures augmented with attention layers, reinforcing the idea that innovation builds on existing foundations rather than discarding them.

At the systems level, modern AI increasingly relies on modularity. Mixture-of-Experts architectures distribute computation across specialized subnetworks, improving scalability and efficiency. These experts are not novel primitives but combinations of Transformers, CNNs, and MLPs. Even cutting-edge Large Language Models depend on classical neural components at their core.

From a scientific perspective, the concept of architectural obsolescence is misguided. Neural networks encode assumptions about the structure of data. When those assumptions align with the problem domain, performance and efficiency follow. When they do not, even the most advanced model will struggle. Progress in artificial intelligence is therefore best understood as specialization and integration, not replacement. Classical networks, modern architectures, and contemporary paradigms such as LLMs, MoEs, VLMs, SLMs, MLMs, and SAMs each occupy a distinct and essential niche. Modern AI systems succeed not because one architecture dominates, but because multiple architectures are combined intelligently.

LLM – Large Language Model

What it is:

• A neural network, usually Transformer-based, trained on massive text corpora.
  What it does:

• Generates and understands text, answers questions, translates languages, summarizes, and can even reason over long documents.
  Relation to classical models:

• Uses MLPs in feed-forward layers and attention mechanisms. Transformers are built on classical building blocks.

LCM – Latent Concept Model

What it is:

• A model designed to extract and represent hidden (latent) concepts from data, often in an interpretable or structured latent space.
  What it does:

• Learns abstract features or concepts that may not be directly observable, useful in tasks like recommendation systems or multimodal reasoning.
  Relation to classical models:

• Often uses MLPs, autoencoders, or CNNs to extract latent features.

LAM – Language-Aware Model

What it is:

• A model that incorporates language understanding into other tasks, e.g., reasoning over text plus another modality.
  What it does:

• Enhances models in vision, graphs, or multimodal tasks by integrating textual context.
  Relation to classical models:

• Combines MLPs, CNNs, Transformers, depending on the primary input type.

MoE – Mixture-of-Experts

What it is:

• A modular architecture with multiple “expert” subnetworks, where only a subset is active for each input.
  What it does:

• Improves scalability and efficiency, allowing very large models without fully computing every parameter for each input.
  Relation to classical models:

• Each “expert” can be an MLP, CNN, or Transformer block—MoE is a system-level design, not a new primitive.

VLM – Vision-Language Model

What it is:

• A model that combines visual input (images/video) with textual input.
  What it does:

• Can answer questions about images, generate captions, or reason across modalities.
  Relation to classical models:

• Uses CNNs or Vision Transformers for images, MLPs in classification heads, and attention layers for multimodal reasoning.

SLM – Sequence Learning Model

What it is:

• A model designed specifically for temporal or sequential data (time series, language, sensor streams).
  What it does:

• Learns patterns over sequences for prediction, anomaly detection, or control tasks.
  Relation to classical models:

• May use RNNs, LSTMs, GRUs, or Transformers depending on scale and latency requirements.

MLM – Masked Language Model

What it is:

• A model trained to predict missing tokens in text, e.g., BERT.
  What it does:

• Learns contextual embeddings and deep language understanding.
  Relation to classical models:

• Transformer-based, with MLPs in feed-forward layers for token prediction.

SAM – Segment Anything Model

What it is:

• A universal image segmentation model that can identify objects in images with minimal guidance.
  What it does:

• Can segment any object in an image, supports zero-shot segmentation for unseen classes.
  Relation to classical models:

• Typically uses CNN backbones for feature extraction combined with Transformers for reasoning and mask prediction.

Recurrent Neural Network (RNN) is a type of deep learning model designed to process sequential data, such as text, speech, or time-series, by utilizing internal feedback loops as memory to remember previous inputs. Unlike feedforward networks, RNNs update their hidden state based on current input and past states, enabling them to make predictions based on data order. 

Training Techniques in PyTorch: Early Stopping, Dropout, and Regularization

For our example, we use the Fashion-MNIST dataset, although the specific dataset is not particularly important for demonstrating these concepts.

Initial Setup

We begin by importing the required libraries and loading our dataset. The model is configured to train for eight epochs, which represents the maximum number of epochs the training process is allowed to run.

However, thanks to early stopping, the training may end before reaching this limit.

Implementing Early Stopping in PyTorch

Since PyTorch does not provide built-in early stopping functionality, we must implement it manually within the training loop.

Tracking the Best Validation Loss

Early stopping is based on monitoring the validation loss. We start by initializing the best validation loss to infinity. This guarantees that the first computed validation loss will always be an improvement.

Defining an Improvement Threshold

Not every improvement is meaningful. To avoid reacting to very small fluctuations, we define a minimum improvement threshold equal to 0.001. If the validation loss fails to improve by at least this value, the epoch is considered below the performance threshold.

Setting Patience

The patience parameter determines how many times the validation loss is allowed to fall below the threshold before training is stopped. In this example, patience is set to two, meaning that training will stop early if the validation loss does not sufficiently improve two times.

Early Stopping Logic in the Training Loop

At the end of each epoch, we compute the validation loss and calculate the difference between the best validation loss and the current validation loss. If the current validation loss is lower, we update the best value. If the improvement does not meet the threshold, a counter is increased. When this counter reaches the patience value, the training loop is interrupted.

This logic allows the model to stop training once it stops learning meaningful patterns.

Dropout for Regularization

Another important technique covered in this demo is dropout, which helps prevent overfitting. In PyTorch, dropout is implemented as a layer, where the parameter p represents the probability that a neuron is zeroed out.

With p set to 0.5, half of the inputs to the dropout layer are randomly set to zero during training. One advantage of PyTorch’s implementation is that the same dropout layer can be reused throughout the model without worrying about input or output sizes.

Regularization and Momentum in the Optimizer

Both L2 regularization and momentum are implemented directly in the optimizer. Regularization is controlled using the weight decay parameter, while momentum is controlled using the momentum parameter.

When using stochastic gradient descent, these options help stabilize training and improve the model’s ability to generalize.

Putting Everything Together

All of these techniques—early stopping, dropout, regularization, and momentum—can be used simultaneously in a single training pipeline.

After training the model, we observe that all eight epochs are completed, even though early stopping is enabled. This happens because the validation loss only fails to improve beyond the threshold in Epoch 6 and Epoch 8. Since the patience value is two, training is allowed to continue.

If the model had been configured to train for ten epochs instead of eight, early stopping would have triggered at Epoch 8