" MicromOne

Pagine

When an Index Is Not the Right Solution for a Slow Query

 When a database query is slow, one of the first solutions that often comes to mind is adding an index.

Indexes are extremely useful. They can dramatically improve query performance by allowing the database to find rows without scanning an entire table. But an index is not a universal solution to every performance problem.

In some situations, adding an index can introduce more costs than benefits. The right optimization depends on the workload, query frequency, data characteristics, and role of the database.

Let's look at some alternatives.

Indexes Have a Cost

Indexes are not free.

Every additional index requires storage, and the database must keep that index up to date whenever the underlying data changes. This means that INSERT, UPDATE, and DELETE operations can become more expensive as the number of indexes increases.

For example, imagine a table that receives thousands of writes every minute, but a particular query runs only once a week as part of an analytical report.

Adding an index specifically to optimize that weekly query might not be worthwhile. The index would be maintained continuously, even though the query that benefits from it is executed only occasionally.

This is an important principle:

The benefit of an index should be evaluated against its maintenance cost and the frequency of the workload that uses it.

An index can be an excellent solution for a frequently executed application query while being a poor choice for an occasional analytical query.

Separate Analytical Workloads from Application Workloads

Another option is to separate analytical workloads from the database that serves the application.

Application databases are usually optimized for transactional workloads: frequent reads and writes, short queries, and predictable response times.

Analytical queries can be very different. They may scan millions of rows, perform aggregations, join large tables, or process historical data.

Running these queries directly against the production database can consume CPU, memory, disk I/O, and other resources that the application needs.

One possible architecture is to provide analysts with a separate copy or replica of the data.

For example:

                  +------------------+
                  | Production DB    |
                  |                  |
Application ----> | Transactions     |
                  +--------+---------+
                           |
                           | Replication
                           v
                  +------------------+
                  | Analytics DB     |
                  |                  |
                  | Reports / BI     |
                  +------------------+

In this model, the production database remains focused on application traffic, while analytical queries can run against a separate system.

Depending on the requirements, this could involve a read replica, a reporting database, a data warehouse, or another dedicated analytical system.

The important idea is not necessarily the specific technology. It is workload isolation.

Background Processing Can Be a Better Approach

Sometimes the best solution is not to make the query faster at all.

Instead, ask whether the calculation really needs to happen when the user requests it.

Suppose an analytical report requires several expensive queries and calculations, but the underlying data changes relatively slowly.

Rather than calculating everything every time an analyst opens the report, the system could perform the calculations periodically in the background.

For example:

Production Data
      |
      v
Background Job
      |
      v
Precomputed Results
      |
      +----> Reporting Table
      |
      +----> CSV / Export
      |
      +----> Dashboard

A scheduled job could run every night, every hour, or according to whatever frequency makes sense for the business.

The results could then be stored in a reporting table or exported to a format such as CSV.

When an analyst needs the information, they are reading already-computed results instead of triggering an expensive calculation against the production data.

This approach is particularly useful when freshness requirements are limited.

If a report only needs to be updated once per day, there may be little value in spending significant resources making a real-time query extremely fast.

Large Text Fields Are a Different Problem

Indexes are also highly dependent on the type of data being searched.

Consider a table containing a large text column:

CREATE TABLE articles (
    id BIGINT PRIMARY KEY,
    title TEXT,
    content TEXT
);

A common requirement might be to search for articles containing particular words or phrases.

A traditional B-tree index is generally not the right tool for arbitrary searches inside large blocks of text.

For example:

SELECT *
FROM articles
WHERE content LIKE '%database performance%';

The problem here is that the database is being asked to find text occurring anywhere inside the value.

This is fundamentally different from looking up an exact value or performing a range comparison.

For this type of workload, full-text search is usually a more appropriate solution.

Full-Text Search

A full-text search system processes text differently from a traditional index.

Instead of treating the entire text value as one large string, the system analyzes the content and builds a searchable representation based on words or other linguistic units.

PostgreSQL, for example, provides built-in full-text search capabilities.

A simplified example might look like:

SELECT *
FROM articles
WHERE to_tsvector('english', content)
      @@ plainto_tsquery('english', 'database performance');

This allows PostgreSQL to use mechanisms designed specifically for searching text rather than relying on a conventional B-tree index.

The general lesson is important:

The type of search you need should influence the type of index or search technology you choose.

When a Specialized Search Engine Makes Sense

For some applications, text search becomes important enough that a dedicated search system is worth considering.

Technologies such as Elasticsearch are designed specifically for search workloads and provide capabilities such as full-text search, relevance scoring, filtering, aggregations, and distributed indexing.

A common architecture might look like this:

              +------------------+
              | Application      |
              +--------+---------+
                       |
             +---------+---------+
             |                   |
             v                   v
       +-----------+       +-------------+
       | Database  |       | Elasticsearch|
       |           |       |             |
       | Source of |       | Search      |
       | Truth     |       | Index       |
       +-----------+       +-------------+

The database remains the authoritative source of the application's data, while the search engine maintains a representation optimized for search.

Of course, introducing another system also introduces additional operational complexity. Data synchronization, indexing pipelines, monitoring, backups, and failure handling all need to be considered.

So a specialized search engine should be introduced because the workload requires it—not simply because a query happens to be slow.

Start With the Workload, Not the Index

When a query is slow, the first question should not always be:

"Which index should I add?"

A better set of questions is:

  • How often does this query run?

  • Is it part of the application's critical path?

  • How much data does it process?

  • Is the workload transactional or analytical?

  • How expensive will an additional index be to maintain?

  • Does the query need real-time results?

  • Would precomputing the result be sufficient?

  • Is the data being searched structured data or large amounts of text?

  • Would a separate reporting or search system be more appropriate?

These questions help identify the actual nature of the problem.

The Broader Lesson

Indexes are one of the most important tools available for database optimization, but they are only one tool among many.

A slow query might be best addressed with an index. In another situation, the right solution could be a read replica, a reporting database, background processing, precomputed results, full-text search, or a dedicated search engine.

The key factors include:

  1. Query frequency — A query executed thousands of times per hour has different optimization requirements from a weekly report.

  2. Workload criticality — Queries on the application's critical path may require very different optimization strategies from occasional analytical queries.

  3. Index maintenance costs — Additional indexes consume storage and can increase the cost of writes.

  4. Workload isolation — Analytical queries may be better executed outside the production database.

  5. Data characteristics — Searching structured values is different from searching large amounts of natural language text.

  6. Freshness requirements — If results do not need to be real-time, background processing and precomputation may be more efficient.

Database performance optimization is rarely about finding a single universal trick.

Adding an index can be the right solution—but only when the workload justifies it. For occasional analytical queries, an index may provide little benefit compared with its ongoing maintenance cost. For heavy reporting workloads, separating analytics from production may be more appropriate. For expensive calculations, background processing and precomputed results can eliminate unnecessary work. And for large-scale text search, full-text indexing or a specialized search engine may be a better fit.

The broader principle is simple:

Optimize for the workload, not just for the query.

Understanding how the database is used, how frequently operations run, how fresh the results need to be, and what kind of data is being searched will usually lead to a better solution than automatically adding another index.

Building, Packaging, and Embedding a React Sports Blog in Microsoft Dynamics 365Building, Packaging, and Embedding a React in Microsoft Dynamics 365

 

Modern enterprise applications increasingly require rich, interactive user experiences that go beyond the capabilities of traditional HTML pages and form customizations. Within Microsoft Dynamics 365 and Power Platform environments, organizations often need advanced interfaces for dashboards, portals, selectors, reporting tools, knowledge bases, and content-driven applications.

React has emerged as one of the most widely adopted frontend libraries for building complex user interfaces due to its component-based architecture, efficient rendering engine, and extensive ecosystem. Integrating a React application into Dynamics 365 enables developers to leverage modern frontend development practices while continuing to benefit from Dataverse, Model-Driven Apps, security roles, business processes, and enterprise governance.

This article presents a technical overview of how a React-based sports blog application can be designed, built, packaged, and embedded into a Dynamics 365 environment using HTML Web Resources and JavaScript integration patterns.


React Architecture Overview

React is a declarative JavaScript library that builds user interfaces through reusable components.

Rather than manipulating the Document Object Model (DOM) directly through imperative code, React maintains a Virtual DOM representation and computes the minimum set of changes required to synchronize the browser interface with application state.

A simplified architecture can be represented as follows:

Application State
        │
        ▼
 React Components
        │
        ▼
   Virtual DOM
        │
        ▼
  Diff Algorithm
        │
        ▼
 Browser DOM

This approach minimizes unnecessary browser reflows and repaints while improving maintainability and scalability.

For a sports blog application, React components may represent:

SportsBlog
│
├── Header
├── ArticleList
│   ├── ArticleCard
│   ├── ArticleCard
│   └── ArticleCard
│
├── FeaturedArticle
│
├── LeagueStandings
│
├── StatisticsWidget
│
└── Footer

Each component is independently developed, tested, and maintained.


Designing the Sports Blog Application

The sports blog serves as a practical example because it combines several common frontend requirements:

  • Large amounts of formatted content
  • Dynamic article rendering
  • Media-rich experiences
  • Real-time updates
  • Reusable UI components
  • Responsive layouts

A typical article object may follow the structure below:

const article = {
    id: "1",
    title:
        "How Analytics Is Transforming Modern Football",
    author:
        "Sports Editorial Team",
    category:
        "Football",
    publishDate:
        "2026-09-25",
    imageUrl:
        "/images/football.jpg",
    content: [
        "Paragraph 1...",
        "Paragraph 2...",
        "Paragraph 3..."
    ]
};

This model separates data from presentation and enables components to remain reusable across multiple scenarios.


Application Initialization

React applications begin by creating a root node that attaches the component tree to a browser DOM element.

<div id="root"></div>

The application entry point mounts the React tree:

import React from "react";
import { createRoot }
    from "react-dom/client";
import App from "./App";
const container =
    document.getElementById("root");
const root =
    createRoot(container);
root.render(
    <App />
);

The root component acts as the orchestration layer for all subsequent views.


Component Development Strategy

A scalable React project should separate concerns between:

Presentation Components

Responsible only for rendering.

function ArticleTitle({ title }) {
    return <h1>{title}</h1>;
}

Container Components

Responsible for:

  • API communication
  • State management
  • Data transformation
  • Business logic

function ArticleContainer() {
    const [article,
        setArticle] = useState();
    useEffect(() => {
        loadArticle();
    }, []);
    return (
        <ArticleView
            article={article}
        />
    );
}

This separation improves maintainability and facilitates testing.


State Management Considerations

As the application grows, state becomes increasingly important.

Typical state categories include:

UI State
├── Modal Open
├── Theme
└── Loading Indicators
Business State
├── Articles
├── Teams
├── Results
└── Statistics
Session State
├── User Preferences

For small projects:

useState()
useReducer()

are sufficient.

For enterprise-scale solutions:

Redux
Zustand
Context API
Recoil

may provide better scalability.


Routing Architecture

If multiple sports sections are required, the application can implement routing.

/
├── football
├── basketball
├── volleyball
├── tennis
└── cycling

Using React Router:

<Route
    path="/football"
    element={<FootballPage />}
/>
<Route
    path="/cycling"
    element={<CyclingPage />}
/>

This allows a single build to support multiple content experiences.


Styling Strategy

Enterprise applications typically avoid inline styles.

A common structure is:

src
│
├── components
├── hooks
├── services
├── pages
│
└── styles
    ├── globals.css
    ├── article.css
    └── widgets.css

Benefits include:

  • Reusability
  • Theme support
  • Better maintainability
  • Easier accessibility compliance


Build Process

The build stage transforms development code into optimized production assets.

Source:

JSX
ES6 Modules
CSS
Images

Compilation:

Vite / Webpack
        │
        ▼
 Minification
 Tree Shaking
 Bundling
        │
        ▼
 Production Assets

Execution:

npm run build

Generated output:

dist/
├── index.html
├── assets/
│   ├── index.js
│   ├── vendors.js
│   └── index.css

The generated files are static and can be hosted by virtually any web platform.


Packaging for Dynamics 365

Dynamics 365 cannot directly execute a React project structure.

Instead, it consumes the generated build artifacts.

Example structure:

dps_/pages/
└── sports_blog/
    ├── index.html
    ├── assets/
    │   ├── index.js
    │   └── index.css

These files become:

HTML Web Resource
JavaScript Web Resource
CSS Web Resource

within the solution.

After publication, they become accessible inside the Model-Driven App runtime.


Opening the React Application Through navigateTo

The most decoupled integration approach consists of launching the React application inside a modal dialog.

let pageInput = {
    pageType:
        "webresource",
    webresourceName:
        "dps_/pages/sports_blog/index.html",
    data:
        JSON.stringify(payload)
};
let navigationOptions = {
    target: 2,
    width: {
        value: 1100,
        unit: "px"
    },
    height: {
        value: 700,
        unit: "px"
    },
    position: 1
};
Xrm.Navigation.navigateTo(
    pageInput,
    navigationOptions
);

Advantages:

  • Loose coupling
  • Independent deployment
  • Reusability
  • Easier maintenance


Passing Data to the React Application

Typically the hosting form sends contextual information.

Example:

{
    recordId:
        currentRecordId,
    ownerId:
        ownerId,
    sectorId:
        sectorId,
    campaignId:
        campaignId
}

Within React the payload is parsed:

const params =
    new URLSearchParams(
        window.location.search
    );
const data =
    JSON.parse(
        params.get("data")
    );

This creates a communication bridge between Dynamics and React.


Embedding React Directly Into a Form

A more integrated alternative consists of placing the Web Resource directly inside the form designer.

Architecture:

Dynamics Form
│
├── Standard Fields
│
├── Subgrid
│
└── React Web Resource

Communication occurs through:

const contentWindow =
    await control
        .getContentWindow();

Data can then be injected:

contentWindow.setSportsArticle({
    title:
        "Advanced Football Analytics",
    author:
        "Editorial Team"
});

This approach enables real-time synchronization between Dataverse records and React components.


Security Considerations

Enterprise deployments should consider:

Input Validation

Never trust incoming data.

if (!article.title) {
    throw new Error(
        "Invalid article"
    );
}

XSS Protection

Avoid:

dangerouslySetInnerHTML()

unless content is sanitized.

Environment Isolation

Deploy through managed solutions rather than manually modifying production assets.


Performance Optimization

Large React applications may suffer from excessive rendering.

Common optimization techniques include:

Memoization

useMemo()

Callback Optimization

useCallback()

Code Splitting

React.lazy()

Dynamic Imports

import(
    "./HeavyComponent"
);

Asset Compression

Gzip
Brotli

These strategies become increasingly important when the application is embedded inside enterprise solutions where multiple components may be executed simultaneously.


Future Evolution: PCF vs Web Resources

While HTML Web Resources remain a valid integration mechanism, modern Power Platform development increasingly adopts:

Power Apps Component Framework (PCF)

Advantages include:

  • Native Dataverse integration
  • Strong typing
  • Lifecycle management
  • Better form integration
  • Modern React support

For simple content portals and blog experiences, HTML Web Resources remain highly effective.

For enterprise-grade reusable controls, PCF often becomes the preferred long-term solution.

Integrating React into Dynamics 365 enables organizations to combine modern frontend engineering practices with Microsoft’s enterprise platform capabilities. A React sports blog serves as an excellent reference implementation because it demonstrates component design, state management, routing, build optimization, packaging, deployment, and form integration patterns.

By compiling the application into static build artifacts and deploying them as Dynamics 365 Web Resources, developers can deliver sophisticated user experiences while preserving compatibility with Dataverse, Model-Driven Apps, business processes, and enterprise governance requirements. As Power Platform continues to evolve, this architecture provides a scalable foundation that can later transition into PCF-based solutions while maintaining the same React development paradigm.

How to Remove Visual Studio Workspace Files from a Git Repository

When working on a software project with Visual Studio and Git, development environments can generate workspace and user-specific files that do not belong in the source repository.

Keeping these files under version control can create unnecessary changes, merge conflicts, and clutter. In this guide, we’ll look at how to remove Visual Studio workspace files that have already been added to Git.

Why Remove Visual Studio Workspace Files?

Visual Studio creates various files and folders to store local development settings, workspace information, and user-specific configuration.

These files are often specific to a single developer's machine and generally should not be shared through Git. A common example is the .vs folder.

Instead of committing these files, it is usually better to add them to .gitignore so Git ignores them in the future.

Removing the .vs Folder from Git

If the .vs folder has already been tracked by Git, simply adding it to .gitignore is not enough. Git will continue tracking files that are already part of the repository.

The following command removes the .vs folder from Git's index while keeping the files on your local computer:

git rm -r --cached .vs

The --cached option is important because it tells Git to stop tracking the files without deleting your local copy.

Commit the Change

After removing the workspace files from Git's index, create a commit describing the change:

git commit -m "Remove Visual Studio workspace files"

This records the change in your local Git history.

Push the Changes

Finally, push the commit to the remote repository:

git push

After the push completes, the .vs files will no longer be tracked in the remote repository.

Add .vs to .gitignore

To prevent the same files from being added again, make sure your .gitignore file contains:

.vs/

This tells Git to ignore the Visual Studio .vs directory in the future.

A typical Visual Studio project may also use a .gitignore containing other generated files, depending on the project type.

Complete Workflow

The basic workflow is:

git rm -r --cached .vs
git commit -m "Remove Visual Studio workspace files"
git push

And in .gitignore:

.vs/

Removing Visual Studio workspace files from Git is a simple cleanup task that can make a repository easier to maintain. The key point is that .gitignore only prevents untracked files from being added; it does not automatically remove files that Git is already tracking.

Using git rm -r --cached .vs, followed by a commit and push, removes the workspace files from version control while keeping them available locally for Visual Studio.

This approach helps keep your repository focused on the actual source code and project files rather than temporary, generated, or developer-specific data.





Working with JSON and Other Data Types in PostgreSQL

PostgreSQL is much more than a relational database that stores simple values such as numbers, strings, and dates. One of its most powerful features is its support for advanced data types, including JSON, arrays, geometric data, and geographic coordinates.

In this article, we will focus on JSON and see how PostgreSQL can work directly with data stored inside a JSON document.

Working with JSON in PostgreSQL

JSON, which stands for JavaScript Object Notation, is a popular format for representing structured data.

For example, a JSON value might look like this:

{
  "name": "Alice",
  "age": 30
}

Instead of storing name and age as separate database columns, we can store the entire structure in a PostgreSQL column using the JSON data type.

Imagine that we have a table called json_test with a column called val:

CREATE TABLE json_test (
    val JSON
);

The table could contain values such as:

{"name": "Alice", "age": 30}
{"name": "Bob", "age": 25}

The interesting part is that PostgreSQL doesn't simply treat these values as ordinary text. It understands the JSON structure and allows us to access individual properties.

Extracting Values from JSON

PostgreSQL provides special operators for working with JSON.

For example, we can use the -> operator to extract a property:

SELECT val -> 'name'
FROM json_test;

This tells PostgreSQL to take the val column, interpret it as JSON, and extract the name property.

The result would be something similar to:

"Alice"
"Bob"

There is also another very useful operator: ->>.

SELECT val ->> 'name'
FROM json_test;

The difference is important:

  • -> returns the result as JSON.

  • ->> returns the result as text.

Therefore, if you need to work with the extracted value as a normal text value, ->> is often the appropriate choice.

Using JSON Values in a WHERE Clause

PostgreSQL can also use values inside JSON documents when filtering rows.

For example:

SELECT val
FROM json_test
WHERE val ->> 'name' = 'Alice';

Here, name is not a database column.

It is a property contained inside the JSON value stored in the val column.

PostgreSQL goes inside the JSON document, extracts the value associated with name, and compares it with the string 'Alice'.

The result will therefore contain only the JSON object belonging to Alice:

{
  "name": "Alice",
  "age": 30
}

This demonstrates an important feature of PostgreSQL: SQL queries can operate not only on traditional database columns but also on structured data stored inside those columns.

Why JSON Support Is Useful

The ability to query JSON directly can be extremely useful when dealing with data whose structure is flexible or changes over time.

For example, an application might receive information from an external API:

{
  "name": "Alice",
  "email": "alice@example.com",
  "preferences": {
    "language": "English",
    "notifications": true
  }
}

Instead of immediately transforming every property into separate relational columns, PostgreSQL can store the JSON structure and allow the application to query individual properties when necessary.

This can be particularly useful for:

  • API responses

  • Configuration data

  • Metadata

  • Semi-structured information

  • Data with optional properties

  • Applications where the structure can evolve over time

PostgreSQL Supports More Than JSON

JSON is only one example of PostgreSQL's support for advanced data types.

PostgreSQL also provides support for data types such as:

  • Arrays

  • Geometric types

  • Spatial and geographic data

  • JSON and JSONB

  • Dates and timestamps

  • Network address types

  • Range types

This means that PostgreSQL can represent and manipulate much more complex information than simple strings and numbers.

JSON Is More Than Just Text

One of the most important concepts to understand is that PostgreSQL can actually understand the structure of JSON data.

When we execute:

SELECT val ->> 'name'
FROM json_test;

PostgreSQL is not simply searching for the word name inside a string.

It understands that name is a property of a JSON object, accesses that property, extracts its value, and returns it to the query.

This makes JSON a powerful option when relational and semi-structured data need to coexist in the same database.

PostgreSQL's support for advanced data types is one of the features that makes it such a powerful database system.

JSON is a particularly good example because it allows developers to combine the reliability and querying capabilities of a relational database with the flexibility of a structured document format.

With operators such as and , we can easily extract information from JSON documents and even use that information in WHERE clauses.

And JSON is only the beginning. PostgreSQL provides many other specialized data types and operators that allow developers to work with complex data directly inside the database.

Understanding these capabilities is an important step toward making better use of PostgreSQL and its extensive feature set.

Origin Pilot China's First Quantum Computer Operating System Now Available for Download

The quantum computing landscape is shifting from experimental hardware to scalable, full-stack software ecosystems. In a groundbreaking milestone for global quantum development, the Origin Pilot quantum computing operating system is officially available for public online download on the Origin Quantum Cloud Platform. 

First introduced in 2021 by Origin Quantum, Origin Pilot serves as the central orchestration and scheduling hub for quantum processors. Now in its fourth major iteration, this release marks the world's first complete quantum operating system made available for local deployment and public download, breaking away from the cloud-only access models typical of Western programming frameworks. 
Deployed natively on the Origin Wukong series of superconducting quantum computers, Origin Pilot acts as the "soft heart" of the quantum ecosystem, bridging the gap between sophisticated quantum hardware and real-world industrial applications. 

Why Origin Pilot Matters: The Power of Local Deployment

Unlike cloud-constrained developer tools, Origin Pilot provides a standardized driving system and unified programming interfaces. This allows research institutions, universities, and enterprise developers worldwide to download the OS locally, integrate quantum control directly within their labs, or connect seamlessly to physical quantum chips. 

Core System Capabilities

  • Unified Multi-Backend Access: Origin Pilot provides cross-platform compatibility across diverse physical architectures—including superconducting circuits, trapped ions, semiconductors, neutral atoms, and photonics. It manages high-efficiency data channels and effortlessly coordinates hybrid classical-quantum tasks. 
  • Multi-User & Multi-Task Collaborative Scheduling: The OS handles concurrent multi-user queues, optimizing the collaborative allocation of quantum and classical computing power under heavy workloads. 
  • Automated Qubit Calibration: To counteract environmental decoherence, the system features real-time, automatic qubit calibration and system-level noise correction matrices, drastically improving the operational reliability of sampling results. 
  • Parallel Task Processing: It maximizes execution efficiency by executing multiple quantum tasks in parallel across the underlying hardware.

Tailored Editions for Global Innovators

To accommodate different operational scales, Origin Pilot has been launched in two distinct versions: 
Community Edition: Free to download globally, giving researchers and developers the tools to experiment with quantum algorithms, run simulations, and connect to real backends using independent frameworks like QPanda. 

Enterprise Edition: Adds advanced layers of security, including post-quantum cryptographic (PQC) encryption modules and intelligent error mitigation tools designed for commercial and industrial deployment. Shaping the Future of Quantum Software. 
As quantum technology accelerates under global strategic initiatives, establishing the software standards of tomorrow is critical. By open-sourcing the integration layer of quantum computing, Origin Quantum is lowering the barrier to entry, inviting global developers to collaborate on building a truly open quantum architecture. 
Whether you are looking to run advanced quantum simulations via Python interfaces, configure a modular quantum computer in a university lab, or test cutting-edge algorithms on China’s third-generation superconducting hardware, Origin Pilot provides the foundational infrastructure. 

Get Started Today

Experience the future of quantum operating systems firsthand.
Download Origin Pilot on the Official Origin Quantum Cloud Platform to start programming, compiling, and executing tasks on live quantum architectures. 

How to Improve SQL Query Performance When Working with Large Datasets

As databases grow from thousands to millions—or even billions—of rows, SQL query performance becomes increasingly important.

A query that runs in a fraction of a second on a small table can become painfully slow when executed against a large production dataset. The good news is that many performance problems can be addressed with a simple principle:

Do less work, as early as possible.

This article explores some of the most useful techniques for improving SQL query performance, particularly when working with large datasets.

Filter Data as Early as Possible

One of the simplest ways to improve query performance is to reduce the number of rows that need to be processed.

Suppose you have a large events table containing years of time-series data, but you only need events from January 2026:

SELECT *
FROM events
WHERE event_date >= '2026-01-01'
  AND event_date < '2026-02-01';

Instead of processing the entire table, the database can work with a much smaller subset of data.

This becomes especially valuable when the filtered data is later:

  • Joined with other tables

  • Aggregated

  • Sorted

  • Deduplicated

  • Transformed

The fewer rows entering these operations, the less work the database has to perform.

A useful development strategy

When developing a query against a very large table, start with a small subset of the data.

For example:

  1. Explore a small time range.

  2. Develop and test your query.

  3. Verify that the results are correct.

  4. Remove the restriction.

  5. Run the final query against the full dataset.

This makes development faster and can help you identify logical problems without repeatedly processing millions of rows.

Don't Assume LIMIT Makes Aggregations Faster

LIMIT is useful for restricting the number of rows returned, but it does not necessarily reduce the amount of work required by an aggregation.

Consider:

SELECT COUNT(*)
FROM events
LIMIT 10;

The result of COUNT(*) is already a single row. The database still has to calculate the count before it can apply the LIMIT.

The same principle applies to grouping:

SELECT customer_id, COUNT(*)
FROM events
GROUP BY customer_id
LIMIT 10;

Conceptually, the database needs to perform the grouping before it knows which aggregated rows can be returned.

If your goal is to reduce the amount of input data during testing, you can put the LIMIT inside a subquery:

SELECT COUNT(*)
FROM (
    SELECT *
    FROM events
    LIMIT 1000
) AS sample;

This genuinely reduces the number of rows being aggregated.

However, there is an important warning: the result now represents only the sample of 1,000 rows, not the complete dataset.

This technique is excellent for testing query logic, but it should not be confused with an optimization that preserves the original result.

Reduce Table Sizes Before Joining

Joins can become expensive when they involve large datasets.

Imagine two tables:

  • accounts

  • web_events

Suppose web_events contains hundreds of millions of records, but your final result only needs the number of events per account.

Instead of joining every event to the accounts table first, you can aggregate the events before performing the join:

SELECT account_id, COUNT(*) AS event_count
FROM web_events
GROUP BY account_id;

This produces a much smaller result.

You can then join that result with accounts:

SELECT
    a.name,
    e.event_count
FROM accounts AS a
JOIN (
    SELECT
        account_id,
        COUNT(*) AS event_count
    FROM web_events
    GROUP BY account_id
) AS e
    ON a.id = e.account_id;

The key idea is:

Reduce the number of rows before expensive operations such as joins.

If web_events contains 500 million rows but only 100,000 accounts, the aggregation can potentially reduce the data involved in the subsequent join dramatically.

Be Careful When Joining Multiple Detail Tables

One of the most important SQL performance and correctness issues occurs when joining multiple tables that contain several rows for the same entity.

Consider two tables containing records by date:

DateTable ATable B
Jan 1100 rows200 rows
Jan 250 rows100 rows

If you join these tables directly on the date, the database can produce a multiplicative result.

For January 1:

100 × 200 = 20,000 rows

For January 2:

50 × 100 = 5,000 rows

This can quickly become a serious performance problem.

Even more importantly, it can produce incorrect aggregate results.

The better approach: aggregate first

Instead of joining the raw tables, aggregate each one independently:

SELECT
    date,
    COUNT(*) AS a_count
FROM table_a
GROUP BY date;

And:

SELECT
    date,
    COUNT(*) AS b_count
FROM table_b
GROUP BY date;

You can then join these much smaller datasets:

SELECT
    COALESCE(a.date, b.date) AS date,
    a.a_count,
    b.b_count
FROM (
    SELECT
        date,
        COUNT(*) AS a_count
    FROM table_a
    GROUP BY date
) AS a
FULL JOIN (
    SELECT
        date,
        COUNT(*) AS b_count
    FROM table_b
    GROUP BY date
) AS b
    ON a.date = b.date;

Instead of joining potentially millions of raw records, you're joining a result containing approximately one row per date.

This pattern is particularly powerful when working with large event, transaction, or log tables.

Understand Why COUNT(DISTINCT ...) Can Matter

Join multiplication can also lead to incorrect counts.

Imagine a sales representative has:

  • 5 orders

  • 20 web events

If you join the two detail tables on the sales representative, you could end up with:

5 × 20 = 100 rows

A simple:

COUNT(sales_rep_id)

could therefore count the same sales representative many times.

In some situations, you may need:

COUNT(DISTINCT sales_rep_id)

to count each representative only once.

However, COUNT(DISTINCT ...) should not automatically be considered the solution to every join problem.

If the underlying issue is that you're joining large detail tables unnecessarily, pre-aggregation is often a much better approach.

For example:

SELECT sales_rep_id, COUNT(*) AS order_count
FROM orders
GROUP BY sales_rep_id;

And separately:

SELECT sales_rep_id, COUNT(*) AS event_count
FROM web_events
GROUP BY sales_rep_id;

You can then join these compact results.

This avoids creating the potentially enormous intermediate dataset in the first place. 

Use EXPLAIN to Understand What the Database Is Doing

When a query is slow, guessing is rarely the best strategy.

Most SQL databases provide an EXPLAIN command that allows you to inspect the query execution plan.

For example:

EXPLAIN
SELECT ...
FROM ...
WHERE ...;

Depending on the database system, the execution plan can provide information about:

  • The order in which operations are performed

  • Estimated row counts

  • Join strategies

  • Scans and indexes

  • Sort operations

  • Aggregations

  • Estimated costs

The exact syntax and information available vary between database systems, but the underlying idea is the same: understand where the database expects to spend its work.

A practical optimization workflow

A useful process looks like this:

Write query
    ↓
EXPLAIN
    ↓
Identify expensive operations
    ↓
Modify query
    ↓
EXPLAIN again
    ↓
Compare the plans
    ↓
Measure the actual performance

The cost shown by EXPLAIN is generally not a direct measurement of execution time. Think of it as an estimate that helps the optimizer and developer compare alternative execution strategies.

Where supported, commands such as EXPLAIN ANALYZE can provide actual execution information as well.

The Core Principle: Do Less Work Earlier

Most of the techniques discussed above are different applications of the same idea:

The best optimization is often to prevent unnecessary work from happening in the first place.

Think about a query as a pipeline.

If you start with 100 million rows and filter it down to 1 million rows before performing a join, that join has dramatically less data to process.

If you aggregate those 1 million rows into 50,000 groups before joining another table, the next operation becomes even smaller.

In general:

Filter → Aggregate → Join → Transform

is often more efficient than:

Join everything → Transform everything → Aggregate at the end

Of course, SQL optimizers can rewrite queries and choose their own execution strategies, so the database engine may already perform some of these optimizations automatically. Nevertheless, writing queries that clearly express the intended reduction of data can make both performance and correctness easier to reason about.

A Simple Before-and-After Example

Consider a query that joins two large tables directly:

SELECT
    ...
FROM large_table_a AS a
JOIN large_table_b AS b
    ON a.date = b.date
GROUP BY ...;

If both tables contain many records per date, this join can generate a huge intermediate result.

A more efficient pattern is often:

SELECT
    ...
FROM (
    SELECT
        date,
        ...
    FROM large_table_a
    GROUP BY date
) AS a
JOIN (
    SELECT
        date,
        ...
    FROM large_table_b
    GROUP BY date
) AS b
    ON a.date = b.date;

Now each table has been reduced before the join.

Instead of joining potentially millions of detail records, the database may only need to join a relatively small number of aggregated rows.

The result is not only potentially faster—it can also be much easier to reason about.

Performance Is Important, but Correctness Comes First

There is one rule that should always come before performance:

Never optimize a query by accidentally changing what it means.

For example, adding:

LIMIT 1000

may make a query run faster, but it also changes the dataset being analyzed.

Likewise, moving an aggregation before a join can improve performance, but only if the new aggregation still produces the intended result.

Good SQL optimization therefore involves two questions:

  1. Does this query produce the correct result?

  2. Can it produce that result with less work?

Only after correctness has been established should you optimize aggressively.

When SQL queries become slow on large datasets, the solution is often not a complicated trick. It is about reducing unnecessary work.

The most important techniques are:

  • Filter early to reduce the number of rows.

  • Don't rely on LIMIT to speed up aggregations.

  • Aggregate before joining when it is logically correct.

  • Avoid joining multiple large detail tables unnecessarily.

  • Use COUNT(DISTINCT ...) when duplicate rows created by joins would otherwise produce incorrect counts.

  • Use EXPLAIN and execution plans to identify expensive operations.

  • Compare query plans and measure performance after making changes.

  • Always preserve correctness.

The principle worth remembering is simple:

Do less work, as early as possible.

When working with millions or billions of rows, reducing the amount of data before expensive operations such as joins, sorting, and aggregation can make an enormous difference.

Good SQL isn't just about getting the right answer. It's about getting the right answer efficiently.

How to Create a Hover Popup with an Embedded Game Using JavaScript

The technique can be useful in dashboards, CRM interfaces, internal tools, portals, and other web applications where you want to provide additional interactive content without changing the main page layout.


The basic idea is simple:

  1. Find a specific grid or control using a data-control-name attribute.

  2. Wait until the element becomes available in the DOM.

  3. Dynamically create a popup.

  4. Add an iframe containing external content.

  5. Display the popup when the mouse enters the grid.

  6. Hide it when the mouse leaves the grid and the popup is no longer being hovered.

For example, our target element can be identified using:

<div data-control-name="casetypecode">
    ...
</div>

The JavaScript function can then be called with:

disableAllGrids("casetypecode");

Waiting for the Grid to Become Available

One of the first challenges is that the target element may not exist immediately when the JavaScript code runs.

This is particularly common in applications that dynamically generate their interface.

The script solves this by using setInterval():

const interval = setInterval(function () {

    const parentDoc = window.parent.document;

    const grid = parentDoc.querySelector(
        '[data-control-name="' + dataControlName + '"]'
    );

    console.log("Looking for:", dataControlName);
    console.log("Grid:", grid);

    if (!grid) {
        return;
    }

    clearInterval(interval);

    // Continue with the setup...

}, 500);

Every 500 milliseconds, the script checks whether the requested element exists.

Once it finds the element, the interval is stopped with:

clearInterval(interval);

This prevents unnecessary DOM queries from continuing indefinitely.

Accessing the Parent Document

The script uses:

const parentDoc = window.parent.document;

This allows the code to work with the document containing the iframe rather than only the current document.

This technique requires the current page and its parent document to satisfy the browser's same-origin security rules. If they belong to different origins, browser security restrictions can prevent access to window.parent.document.

Creating the Popup Dynamically

Instead of requiring an existing popup in the HTML, the script creates one dynamically:

const popup = parentDoc.createElement("div");

We then configure its appearance using JavaScript:

popup.style.position = "fixed";
popup.style.width = "70vw";
popup.style.height = "70vh";
popup.style.backgroundColor = "#fff";
popup.style.border = "1px solid #ccc";
popup.style.borderRadius = "8px";
popup.style.boxShadow = "0 5px 20px rgba(0,0,0,.3)";
popup.style.zIndex = "999999";
popup.style.display = "none";
popup.style.overflow = "hidden";

A few properties are particularly important.

position: fixed

The popup is positioned relative to the browser viewport rather than the normal document flow.

z-index

popup.style.zIndex = "999999";

A high z-index helps ensure that the popup appears above other interface elements.

display: none

Initially, the popup is hidden:

popup.style.display = "none";

It becomes visible only when the user hovers over the target element.

Embedding External Content with an iframe

The next step is to create an iframe:

const iframe = parentDoc.createElement("iframe");

The iframe can then load the external game or application:

iframe.src = "YOUR_EXTERNAL_URL";

The iframe is configured to occupy the entire popup:

iframe.style.width = "100%";
iframe.style.height = "100%";
iframe.style.border = "0";

Finally, it is added to the popup:

popup.appendChild(iframe);

And the popup itself is added to the page:

parentDoc.body.appendChild(popup);

At this point, the popup exists in the DOM but remains invisible.

Showing the Popup on Mouse Hover

The popup is displayed when the user enters the grid:

grid.addEventListener("mouseenter", function () {

    const rect = grid.getBoundingClientRect();

    popup.style.left = rect.left + "px";
    popup.style.top = (rect.bottom + 10) + "px";

    popup.style.display = "block";
});

The key part is:

grid.getBoundingClientRect();

This returns the position and dimensions of the target element.

The script uses those coordinates to position the popup underneath the grid:

popup.style.left = rect.left + "px";
popup.style.top = (rect.bottom + 10) + "px";

The + 10 adds a small 10-pixel gap between the grid and the popup.

Hiding the Popup

When the mouse leaves the grid, we don't immediately hide the popup.

Instead, the script waits 100 milliseconds:

grid.addEventListener("mouseleave", function () {

    setTimeout(function () {

        if (!popup.matches(":hover")) {
            popup.style.display = "none";
        }

    }, 100);

});

This small delay is important.

Without it, moving the mouse from the grid toward the popup could cause the popup to disappear before the cursor reaches it.

The script checks:

popup.matches(":hover")

If the popup is not being hovered, it is hidden.

Closing the Popup When the Cursor Leaves It

There is also a second event handler:

popup.addEventListener("mouseleave", function () {
    popup.style.display = "none";
});

This ensures that the popup disappears as soon as the user moves the cursor away from it.

Together, the two mouse events create a simple hover interaction:

        Target Grid
             │
        mouse enters
             ↓
       ┌───────────┐
       │   Popup   │
       │           │
       │   iframe  │
       │           │
       └───────────┘
             │
       mouse leaves
             ↓
          Hidden

The Complete Concept

The complete function follows this sequence:

function disableAllGrids(dataControlName) {

    const interval = setInterval(function () {

        const parentDoc = window.parent.document;

        const grid = parentDoc.querySelector(
            '[data-control-name="' + dataControlName + '"]'
        );

        if (!grid) {
            return;
        }

        clearInterval(interval);

        const popup = parentDoc.createElement("div");

        popup.style.position = "fixed";
        popup.style.width = "70vw";
        popup.style.height = "70vh";
        popup.style.backgroundColor = "#fff";
        popup.style.border = "1px solid #ccc";
        popup.style.borderRadius = "8px";
        popup.style.boxShadow = "0 5px 20px rgba(0,0,0,.3)";
        popup.style.zIndex = "999999";
        popup.style.display = "none";
        popup.style.overflow = "hidden";

        const iframe = parentDoc.createElement("iframe");

        iframe.src = "YOUR_EXTERNAL_URL";

        iframe.style.width = "100%";
        iframe.style.height = "100%";
        iframe.style.border = "0";

        popup.appendChild(iframe);
        parentDoc.body.appendChild(popup);

        grid.addEventListener("mouseenter", function () {

            const rect = grid.getBoundingClientRect();

            popup.style.left = rect.left + "px";
            popup.style.top = (rect.bottom + 10) + "px";

            popup.style.display = "block";
        });

        grid.addEventListener("mouseleave", function () {

            setTimeout(function () {

                if (!popup.matches(":hover")) {
                    popup.style.display = "none";
                }

            }, 100);

        });

        popup.addEventListener("mouseleave", function () {
            popup.style.display = "none";
        });

    }, 500);
}

disableAllGrids("casetypecode");

Why Use a Function Parameter?

The function accepts:

dataControlName

This makes the solution reusable.

Instead of hard-coding:

'[data-control-name="casetypecode"]'

the script dynamically creates the selector:

'[data-control-name="' + dataControlName + '"]'

Therefore, you can target different controls simply by changing the function argument:

disableAllGrids("casetypecode");

or:

disableAllGrids("anotherControl");

This is much more flexible than creating a separate function for every grid.

Possible Improvements

Although the basic implementation works, there are several ways to make it more robust.

Use CSS Instead of Inline Styles

For larger projects, it is often cleaner to create a CSS class rather than setting every property with JavaScript.

For example:

.game-popup {
    position: fixed;
    width: 70vw;
    height: 70vh;
    background: #fff;
    border: 1px solid #ccc;
    border-radius: 8px;
    box-shadow: 0 5px 20px rgba(0, 0, 0, .3);
    z-index: 999999;
    overflow: hidden;
}

JavaScript can then simply use:

popup.className = "game-popup";

This makes the JavaScript easier to read and allows designers to modify the appearance without touching the code.

Add a Maximum Width and Height

Using:

width: 70vw;
height: 70vh;

works well for many screens, but the popup can become very large on high-resolution displays.

You could add limits such as:

max-width: 1200px;
max-height: 800px;

This can improve usability on larger monitors.

Consider Mobile Devices

Mouse hover does not exist in the same way on touch devices.

For mobile interfaces, a click or tap-based interaction is usually more appropriate.

For example, you could use:

grid.addEventListener("click", function () {
    // Open popup
});

and provide a dedicated close button.

Prevent Duplicate Popups

If the function is called multiple times for the same control, multiple event listeners and popups could potentially be created.

A production implementation should therefore consider storing a reference to the created popup or marking the grid as already initialized.

For example:

if (grid.dataset.popupInitialized === "true") {
    return;
}

grid.dataset.popupInitialized = "true";

This prevents the same element from being initialized more than once.

Browser Security Considerations

When embedding external content, browser security policies can affect whether the iframe is allowed to load.

The external website may use headers such as X-Frame-Options or Content Security Policy (frame-ancestors) to prevent embedding.

In addition, accessing:

window.parent.document

is subject to the browser's same-origin policy.

Therefore, this technique should always be tested in the actual environment where it will be deployed.

A dynamically generated hover popup is a simple but powerful way to add interactive content to an existing web interface.

The approach demonstrated here combines several useful JavaScript techniques:

  • DOM element detection

  • Dynamic element creation

  • iframe embedding

  • mouse event handling

  • viewport-based positioning

  • delayed hiding

  • reusable functions

  • interaction with a parent document