Baking a batch: Processing multiple requests at once

Computers are built to process requests: they receive commands and carry them out, often so quickly that users barely notice the time involved.

But when you have hundreds, thousands, or millions of requests to make, the bottleneck often isn’t the work itself. It’s how that work gets submitted.

Entering requests individually takes time. 

That’s where batch processing (bulk validation) comes in.

What is batch processing?

This process allows multiple units of work to be collected and processed together rather than requiring a user to initiate each one individually. Depending on the system, that might mean uploading a file containing thousands of records, grouping multiple API operations into fewer requests, or submitting a job that can run with little or no further human involvement.

With the right hardware and software in place, batching can reduce manual effort, reduce repeated overhead, and make large amounts of repetitive work much more manageable.

Why processing slows down at scale

When a computer task takes longer than expected, the delay usually comes from one of two places: the user or the system.

Sometimes the computer is the bottleneck. Games lag.

Other times, the bottleneck is the person sitting at the computer.

Typing, clicking, copying, uploading, and submitting requests all take time. One manual action may be insignificant. Repeat it 10,000 times, however, and suddenly the preparation takes considerably longer than the actual processing.

So there are really two ways to slow down the kitchen: overwhelm the equipment or make the cook prepare everything one ingredient at a time.

The computer bottleneck: Per-request overhead adds up

Computers are fast, but every request still requires some work beyond the task you actually care about.

Depending on the system, a request may need to be created, transmitted, authenticated, interpreted, routed, processed, and returned. Some of those steps take only fractions of a second, but fractions of a second matter when they’re repeated thousands or millions of times.

When work is divided into many individual requests, even if each operation is relatively small, the system may have to repeat surrounding work.

The advantage is that it can often reduce how frequently it has to perform the setup surrounding that work.

Instead of preparing the oven and ingredients for every cookie, you fill the tray.

The human bottleneck: Manual entry doesn’t scale

Computers may run into technical limits, but humans have a different problem: input is slow. That friction is easy to ignore on a small scale.

Renaming one file? Easy.

Renaming 5,000 files by hand? Not so much.

It’s the digital equivalent of measuring chocolate chips individually before putting them into the dough.

How batching reduces the work

There is a way to tell a computer, “That. All of it. I want all of that.”

A familiar example is copy-and-paste.

Copy and paste isn’t technically the same thing as batch processing, but it demonstrates the principle nicely. Instead of re-entering information character by character, you select a larger unit of information and move it together.

Batch processing takes that same basic idea and applies it to computational work.

Instead of requiring someone to manually initiate thousands of repetitive operations, the work is grouped and handed off to the system. Once submitted, many batch jobs can continue with little or no additional user involvement. 

The computer gets work in a form it can handle efficiently, while the human gets to go do literally anything else.

Does batching always make processing faster?

Not necessarily.

Batching is particularly useful for high-volume, repetitive work, but it isn’t automatically the best approach for every task.

Sometimes an individual request needs an immediate response. And batching can introduce its own considerations, such as waiting for enough work to accumulate, deciding how large batches should be, or retrying individual items that fail.

The goal is more than to create the biggest batch possible; it’s to package the work in a way that makes sense for the job.

Sometimes you need one cookie right now. Sometimes you need 10,000 by tomorrow morning.

Where batch processing is used

Batch processing shows up all over the place.

Anytime a program lets you upload an Excel or CSV file containing hundreds or thousands of records rather than entering every record individually, you’re taking advantage of the same basic idea.

That’s much better than knocking on the computer’s door 10,000 separate times.

How batch processing works at Smarty

Address data is a particularly good example of where batching becomes useful.

Validating one address manually isn’t much trouble. Validating 100,000 of them individually is a very different afternoon.

Smarty provides several ways to process address lists in bulk, including a web interface, command-line tools, spreadsheet integrations, and APIs. 

Our command-line interface can support millions of address lookups for large one-off jobs.

You hand us the batch.

We handle the trays, and those trays can move quickly!

Smarty can process more than 25,000 address records per second, or roughly 1.5 million per minute, depending on the implementation and subscription.

Because if you have 10,000 cookies to bake, putting them in the oven one at a time will eventually get the job done.

However, batching is how you get your day back.