Cleaning address databases with Smarty's lead developers

We recently brought together Kent Gividen and Bryan Amundson, two of Smarty's brilliant lead developers, to discuss the importance of maintaining a clean address database and how Smarty's innovative tools can make this process easier and more efficient.

Question: "How should users prepare their address data before using Smarty's tools?"

Kent: "To begin with, if you have a database that you're looking to clean the addresses for, you can dump those addresses into a CSV file. And there are several ways to make that easier to handle and process."

By dumping your addresses into a CSV file, you create a standardized format that can be easily manipulated, analyzed, and cleaned using Smarty's tools. Kent mentioned that there are several ways to make this process easier, which could include organizing the data in specific ways (such as all data in a single cell or broken into components) and using appropriate delimiters like commas or tabs to separate the data fields.

This preparation step is crucial to ensure that the address cleaning tools can effectively process and validate the data. For example, if you use the Google Sheets or Microsoft Excel plugins, your data must be formatted in a single cell.

Kent and Bryan talked about the value of non-developer tools, but they both suggested SmartyList as a great and underutilized tool.

Question: "Can you explain how the SmartyList command-line tool works?"

Bryan: "SmartyList, or the CLI as some people call it, is a utility used to process CSV files through our APIs. You can get SmartyList on our website.’"

Bryan talked about the data available in Smarty's documentation in regards to the CLI. It's essentially a step by step tutorial on getting running with very little coding.

By using the command-line interface, users can efficiently process large batches of address data without needing a graphical user interface. This method is particularly useful for developers and technical users who are comfortable working in a command-line environment.

Question: "Could you elaborate on the different match modes available and their uses?"

Kent: "We have three different kinds of match columns: strict, invalid, and enhanced."

He went on to give detailed explanations of each match type.

Strict Mode: This mode requires that the input address must closely match a known address in Smarty's database. If there are significant discrepancies or missing components, the address will not be validated. This is useful for ensuring high accuracy and confidence in the validation results.

Invalid Mode: In this mode, even if the address doesn’t match exactly, Smarty will attempt to return as much information as possible about the address. This includes partially valid addresses where some components may be correct, but others might be missing or incorrect. It provides a more lenient approach, offering useful data even when the input is imperfect.

Enhanced Mode: This is a more aggressive and comprehensive matching mode that can handle various errors and missing components in the input address. It includes additional data sources beyond the USPS, covering non-postal addresses and providing more accurate geocoding and address details. Enhanced mode may require an additional license due to the extra data and processing involved.

Kent also discussed how the different match modes can be utilized when accessing Smarty’s APIs. He mentioned that developers can set the match mode parameter in their API requests to control the validation behavior. This is particularly useful for integrating address validation into custom applications and services.

Question: "Why is the CLI so fast, and how does it handle large volumes of data?"

Bryan: "It's multi-threaded which dramatically increases throughput. The CLI will start 10 threads, and batch up to 100 US Street queries in each request."

Multi-threading allows the CLI to handle a high volume of addresses concurrently, significantly reducing the time required to process large datasets. This capability is particularly beneficial for users dealing with millions of addresses.

Reducing the number of API calls decreases the latency and the time spent waiting for responses from the server, further speeding up the process.

Question: "What options do non-programmers have to clean their address databases using Smarty?"

Bryan: "You can use the Bulk Address tool on Smarty's website to process an entire file of addresses. We also offer plugins for Google Sheets and Excel."

These options ensure that non-programmers can efficiently clean and validate their address databases without involving others with technical skills. By leveraging Smarty's user-friendly tools, plugins, and support services, users can maintain accurate and reliable address data, enhancing their overall data quality and operational efficiency.