Local US Street Address API

ATTENTION: On May 1, 2023, the local version of the Smarty US Street API (v5.5.x) was released which affects how you launch the program. See Managing the Local API Process for the documentation updates.

This page describes how to download, install, and run a local instance of the US Street API.

Contents

1. Glossary
2. API documentation
3. Prerequisites
4. Minimum system requirements
5. Downloading the packages
6. Installation procedures
7. Managing the local API process
8. Connecting to the local API process
9. Updates
10. Docker and containers
11. Automation
12. Troubleshooting

Glossary

Throughout this document we use the following consistently formatted terms:

• US Street API

  The product with the capabilities you wish to host on your local network.

• us-street-data

  The first of two packages you will download and extract. Contains the necessary data files used by the running program.

• us-street-api

  The second of two packages you will download and extract. Contains the main program and other binary resources.

• us-street-api

  The program that you will execute. Found in the us-street-api package.

API documentation

A local installation of the US Street API performs identically to the cloud version hosted by Smarty. Please refer to the documentation for details about input and output fields.

The main difference between the local and cloud installations lies in the parts of the URL used by clients to establish a connection. (scheme://hostname:port) This will be explained in more detail later.

Prerequisites

Access to local US Street API packages and resources is currently restricted to customers with an Enterprise account. Downloading the packages also requires a valid secret key pair and license for authentication.

• mkdir (to create directories)
• cp (to copy files)
• tar (to extract compressed archives)
• curl (to download packages)
• sudo (to perform certain tasks with root-level privileges)
• ldconfig (to refresh the linker's shared library cache)
• $LD_LIBRARY_PATH environment variable (set to extend the linker's shared library cache)

Minimum system requirements

The US Street API is designed to run on a Linux server that can be reached by any clients you intend to call the service. Responsibility for network and server maintainence (as well as the performance of all other operations detailed in this document) rests with your organization.

The server provisioned to run the local US Street API binaries should match or exceed the following criteria:

• 7 gigabytes of disk space / 5 gigabytes of RAM (25/20 for enhanced version)
• 4 CPU cores
• A relatively recent version of the Linux kernel (basically something that can run compiled Go programs). Anything later than v2.6.32 should function without issues.
• A sufficient number of file descriptors based on the number of CPU cores available on your machine. When you run us-street-api, it will provide a helpful message indicating the minumum number of file descriptors it requires. Run ulimit -n to determine how many are available on your system.

Downloading the packages

Running a local instance of the US Street API requires two packages that are available for download via the Smarty Download API:

• us-street-data

Includes all address data accessed by the us-street-api program in order to serve client requests. Here's an example command you might use to download the package:

curl -L "https://download.api.smarty.com/us-street-api/data/latest.tar.gz?auth-id=YOUR_AUTH_ID&auth-token=YOUR_AUTH_TOKEN" -o us-street-data.tar.gz

• us-street-api

Includes the compiled program (redundantly named us-street-api) and several shared libraries. Here's an example command you might use to download this package:

curl -L "https://download.api.smarty.com/us-street-api/linux-amd64/latest.tar.gz?auth-id=YOUR_AUTH_ID&auth-token=YOUR_AUTH_TOKEN" -o us-street-api.tar.gz

• add-ons

Includes the us-rooftop-geocoding-data. Here's an example command you might use to download this package:

curl -L "https://download.api.smarty.com/addons/us-rooftop-geocoding/latest.tar.gz?license=YOUR_LICENSE&auth-id=YOUR_AUTH_ID&auth-token=YOUR_AUTH_TOKEN" -o us-rooftop-geocoding.tar.gz

Includes the us-enhanced-matching-data. Here's an example command you might use to download this package:

curl -L "https://download.api.smarty.com/addons/us-enhanced-matching/latest.tar.gz?license=YOUR_LICENSE&auth-id=YOUR_AUTH_ID&auth-token=YOUR_AUTH_TOKEN" -o us-enhanced-matching.tar.gz

See the sample script below for more details.

Installation procedures

Downloaded packages are gzipped archives and must be extracted (using the tar command) before they can be used.

mkdir ./data
tar xvf us-street-data.tar.gz -C ./data
tar xvf us-street-api.tar.gz
tar xvf us-rooftop-geocoding.tar.gz -C ./data/decorate
mkdir ./data/enhanced
tar xvf us-enhanced-matching.tar.gz -C ./data/enhanced

Make a note of how many file descriptors can be allocated on your system. If you do not have a sufficient number based on the CPUs in your machine, the program will not load. See the Troubleshooting section.

ulimit -n

Having extracted the contents of the packages you are now ready to register the included shared libraries with the linker. There are three different ways to do this. You only need choose one of the following options!

1. Recommended approach: Set the LD_LIBRARY_PATH environment variable to the extracted lib folder.

   export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:`pwd`/lib

   The above command appends the extracted lib folder to any pre-existing value in the LD_LIBRARY_PATH environment variable. Care should be taken when downloading updated versions of the package on the same machine that you don't unnecessarily append this value again. Managing the environment variables on your machine is your responsibility. Notwithstanding this slight "complexity" we encourage this approach because it doesn't require any changes to existing directories on the machine. But, if this approach isn't working, try one of the following methods.

2. With root privileges (ie. sudo), copy the contents of the extracted lib folder to a folder included in the shared library cache. Commonly configured locations include /usr/lib and /usr/local/lib.

   sudo cp ./lib/* /usr/lib

   Some distributions (like CentOS) do not automatically acknowledge new arrivals in the shared library cache locations and so the cache must be rebuilt by running the ldconfig utility with root privileges:

   sudo ldconfig -v

3. Add the extracted lib folder path to /etc/ld.so.conf (on it's own line) and invoke ldconfig as above. This has the effect of including the lib folder in the shared library cache.

Managing the local API process

• To display explanations of command-line flags and their default values:

./us-street-api -help

• Running the program:

./us-street-api

• When the program is run, it uses the config.json file provided in the data package to configure the data path and listening port. The sample config.json file is as follows:

{
	"plaintext_listen_address": ":8080",
	"data_directory": "./data"
}
			

• By default, it looks for this file in the working directory. The path can be changed by using the -config command-line flag as such:

./us-street-api -config /opt/conf/config.json

• The version of the program is displayed on the first output line during execution. In addition, once the server is running you can display the version with the status endpoint as follows:

curl "http://mysmartyhost/status"

NOTE: Running the us-street-api program starts a process that is designed to run continuously until killed.

Connecting to the local API process

Connecting to the local us-street-api process using TLS is currently not supported. This means that the URL scheme will be http instead of https. We recommend using a private network or a proxy to establish encrypted connections if desired. Also, please note that the hostname for the local installation will not be us-street.api.smarty.com (the examples below use localhost). Finally, the default port for the local installation is 8080 rather than 80.

Once the us-street-api program is running, run the following command from another terminal window to send an actual HTTP request to the process:

curl "http://localhost:8080/street-address" -H 'Content-Type: application/json; charset=utf-8' --data-binary '[{"street": "3214 N University ave Provo UT"}]' | python -m json.tool

If everything is functioning correctly then the output should closely resemble the following JSON object:

[
	{
		"analysis": {
			"active": "Y",
			"dpv_cmra": "Y",
			"dpv_footnotes": "AABBR1",
			"dpv_match_code": "Y",
			"dpv_vacant": "N"
		},
		"candidate_index": 0,
		"components": {
			"city_name": "Provo",
			"delivery_point": "14",
			"delivery_point_check_digit": "0",
			"plus4_code": "4405",
			"primary_number": "3214",
			"state_abbreviation": "UT",
			"street_name": "University",
			"street_predirection": "N",
			"street_suffix": "Ave",
			"zipcode": "84604"
		},
		"delivery_line_1": "3214 N University Ave",
		"delivery_point_barcode": "846044405140",
		"input_index": 0,
		"last_line": "Provo UT 84604-4405",
		"metadata": {
			"carrier_route": "C016",
			"congressional_district": "03",
			"county_fips": "49049",
			"county_name": "Utah",
			"dst": true,
			"elot_sequence": "0016",
			"elot_sort": "A",
			"latitude": 40.27658,
			"longitude": -111.65759,
			"precision": "Zip9",
			"rdi": "Commercial",
			"record_type": "S",
			"time_zone": "Mountain",
			"utc_offset": -7,
			"zip_type": "Standard"
		}
	}
]

Updates

Smarty publishes regular updates to both the us-street-api and us-street-data packages. New releases are announced in our open-source Changelog repository.

Docker and containers

Applications are compiled with a dependency on libc. This means that any container images created must have the libc binary available. In production environments containers based upon officially maintained releases of Debian, Ubuntu, Red Hat, and Fedora base images will work. For containers based upon Alpine Linux, you will need to install the libc dependency. While Alpine Linux is typically used to drastically reduce the size of the base image layers, please remember that the host machine caches these layers so that the cost of downloading the base image is only paid once. The latest official Ubuntu base images are around 25 MB in size.

Automation

What follows is a script that you may use to download, install, and run a local instance of the US Street API. It's Bash. Use it as a starting point for putting in place your own update processes. Your mileage may vary. You're welcome.

#!/bin/bash

# Pro Tip:
#   Replace the placeholder auth values in the `curl` commands
#   below with your own auth-id and auth-token, or with environment
#   variables that contain your own auth-id and auth-token values.

# Download the us-street-api package from the download API:
curl -L "https://download.api.smarty.com/us-street-api/linux-amd64/latest.tar.gz?auth-id=YOUR_AUTH_ID&auth-token=YOUR_AUTH_TOKEN" -o us-street-api.tar.gz

# Download the us-street-api data package from the download API:
curl -L "https://download.api.smarty.com/us-street-api/data/latest.tar.gz?auth-id=YOUR_AUTH_ID&auth-token=YOUR_AUTH_TOKEN" -o us-street-data.tar.gz

# Extract the data package:
mkdir ./data
tar xvf us-street-data.tar.gz -C ./data

# Extract the api package:
tar xvf us-street-api.tar.gz -C .

# Register the shared libraries with the linker's cache:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:`pwd`/lib

# Run the us-street-api:
./us-street-api

Troubleshooting

This section contains some common issues that can arise when running us-street-api

• [ERROR] Failed to establish AMS connection on attempt

The most likely cause of this error is that your available file descriptors are not sufficient. Look carefully at the load message which states how many file descriptors are necessary to run us-street-api. You can adjust this using ulimit -n.

• [ERROR] Could not open initial connection to determine expiration dates.

This error typically means that the data does not exist in the directory you specified in the config.json file with the value of data_directory. Or, the program could not find the config.json file. See Managing the Local API Process