Since we announced our collaboration with the World Bank and more partners to create the Open Traffic platform, we’ve been busy. We already shared an initial technical preview of the OSMLR linear referencing system, and now we’re ready to share another.

OSMLR provides a stable linear-referencing system atop the ever-changing network of roadways in OpenStreetMap. It’s used by the Open Traffic platform to associate statistics like speeds and vehicle counts with roadway segments. And it has many more possible uses, beyond just traffic statistics.

We’re now releasing a preview build of OSMLR segment tiles for the entire planet. In this blog post, we explain more about the hierarchical nature of OSMLR tiles, about the criteria for which OSM ways are used to construct OSMLR segments, and how you can download OSMLR segments for testing now.

OSMLR tile hierarchy levels

Similarly to how the Valhalla stores its routing graph in hierarchical tiles, OSMLR segments are available as geographic tiles at three levels of roadway hierarchy:

• The highway level (0) includes drivable road segments with OSM highway tags: motorway, motorway_link, trunk, trunk_link, primary, and primary_link.
• The arterial level (1) includes drivable road segments with OSM highway tags: secondary, secondary_link, tertiary, and tertiary_link.
• The local level (2) includes drivable road segments with OSM highway tags: unclassified, unclassified_link, residential, and residential_link.

Here are all three levels of OSMLR segments in San Francisco:

And here are all three levels of OSMLR segments in Manila, Philippines (where we’ve previously shown a prototype of Open Traffic in action):

Turning OSM ways into OSMLR segments for traffic applications

OSMLR abstracts away the unnecessary geometric details and boundaries of OpenStreetMap ways. Instead, OSMLR segments summarize the key properties of a meaningful stretch of roadway using “location reference points”, or LRPs. (The concept of LRPs comes from OpenLR.)

We used abstract graphics to illustrate how OSM ways are turned into OSMLR segments in this previous blog post. With the preview build of OSMLR segments that is now available, we can highlight real-world examples of how OSMLR segments are generated for use in the Open Traffic platform:

A road segment represents a section of OSM way(s) between two intersections with other ways.

In some situations, one OSMLR segment may represent multiple OSM ways:

In the above example, one OSMLR segment is drawn in double blue strokes along 7th St. in San Francisco. The OSMLR segment has an ID number of 291923924617. The dotted centerlines indicate OSM ways, with a different color for each OSM way. Note how the OSMLR segment represents the entire stretch of 7th St. between Market St. and Mission St. It joins together multiple OSM ways, which were created in order to provide finer detail for bicycle lane tagging. Those differences are not relevant to OSMLR and its use for Open Traffic. Open this area on OpenStreetMap.org to explore these OSM ways.

In other situations, one OSMLR segment may represent only a portion of one OSM way:

In the above example, the San Francisco-Oakland Bay Bridge is split into multiple OSMLR segments (with a different color for each). The Open Traffic platform will be able to provide different speeds along different lengths of the bridge, so we generate OSMLR segments for each kilometer stretch.

Finally, not every OSM way will be represented by an OSMLR segment at all.

A turn channel is a short OSM link (highway tag has _link at the end) connecting two road segments and is assumed to be at-grade. These cannot connect to motorway and trunk OSM ways cannot fork or connect to other links. Turn channels simplify the path or transition between two roads when turning at an intersection.

An internal intersection is a short road segment that lies between two one-way, separated road segments. It provides a path or transition between two road segments and generally are not separate carriageways but rather are “internal” to an intersection and are provided to connect different road segments within the intersection.

Here we see an example of an internal intersection in San Francisco where Van Ness Avenue and Mission Street meet. For the purposes of collecting traffic data, it’s unnecessary to have OSMLR segments within the intersection. Instead, we store speed/congestion data by attaching it to two OSMLR segments: the OSMLR segment that the driver enters the intersection using and the OSMLR segment that the driver exits the intersection using.

Rules for generating OSMLR segments

Now that you know the basics of where OSMLR segments are defined, let’s review some more specific rules we use for generating OSMLR:

• The road segment must be drivable by automobile or other non-emergency vehicle types (bus, taxi, truck, or high-occupancy vehicle).
• A road segment which allows travel in both directions is assigned two OSMLR segments: one in each direction.
• Road segments on all hierarchies are merged or connected across intersections with only non-drivable road segments.
• Road segments on the highway and arterial levels are merged across intersections with only local hierarchy road segments.
• Road segments on the local level include drivable road segments with the exception of service roads (includes parking aisles, driveways, and other minor roads that connect to services).
• OSMLR segments are not generated for turn channels, roundabouts, and internal intersection road segments (as we illustrated above).
• OSMLR segments longer than 1 kilometer are split into multiple segments such that all OSMLR segments are less than 1 kilometer in length. (See the above example of the San Francisco-Oakland Bay Bridge.)

Merging or connecting road segments allows OSMLR definitions to be longer on average and to better average speed along road segments. Creating average speeds or traversal times along extremely short segments can be very inaccurate, especially when the interval between sampled locations (GPS readings) becomes larger. Including a large number of very short segments also creates much more storage and processing demand. However, very long segments do not permit localization of congestion points so a maximum length of one kilometer was settled on.

We plan to generate statistics on “intersection transition” times along with average speeds along an OSMLR segments. This is our rationale for not including turn channels, roundabouts, and internal intersection road segments. These are generally very short road segments and provide a “transition path” between two other road segments. These road segments can be thought of as connecting or transition segments. Since we plan to model the transition times from one OSMLR segment to another we felt that there was no need to form OSMLR definitions for these connecting road segments, we simply merge across them when performing map-matching of GPS traces to OSMLR segments.

Planet-wide OSMLR

We’ve used the the OSMLR generator application to cut OSMLR segments for the entire planet at all three levels of the hierarchy. Here are statistics about the process:

• OSMLR segments in total: 216,819,246
• OSMLR segments under 20 meters in length: 10,303,995
• average length of OSMLR segments: 79.24 meters
• maximum number of OSMLR segments within a single geographic tile: 344,209
• average number of OSMLR segments per tile: 1,528
• total tiles to cover the planet: 141,888

Not all applications require residential and unclassified road segments (the local level, 2, of the hierarchy). Some benefits of only using OSMLR for the highway (0) and arterial (1) levels include:

• Large reduction in total number of OSMLR segments (~37 million compared to ~217 million)
• Large reduction in number of tiles (files) needed (~13.5 thousand compared to ~141 thousand)
• Longer average segment length (462 meters compared to 79 meters) and many less short segments

OSMLR Update and Migration Process

Now that we’ve successfully created OSMLR segments for the entire planet, the next question will be how we update the segments as the road network in OpenStreetMap changes over time.

As OSM edits are made, the vast majority of OSMLR segments remain unchanged. Even most OSMLR segments where the base road segments change will still be valid as the shape and even the endpoints (intersections) can vary slightly and the OSMLR segment remains valid. (See this previous blog post to learn more about how OSMLR uses location reference points to represent the key properties of a segment, without depending upon the exact geometry of a segment.)

Certain types of changes to OSM will invalidate an OSMLR segment:

• the addition of a new highway or arterial road segment that intersects in the middle of an existing OSMLR segments
• changes where the endpoints of a road segment change above some threshold (to be determined)
• changes where the length of a road segment changes above some threshold (also TBD)

These “breaking” changes will require deprecating the existing OSMLR segment and adding new ones. We are working on a means to mark the lineage and change history of OSMLR segments, so this information will be preserved and available in OSMLR tile files.

We have done some initial tests that show that after six weeks, OSMLR segments still correctly associate to over 99.5% of the underlying OSM road segments. More investigation is needed to see how OSMLR segments “degrade” over time and how frequently they should be updated. We need to balance the cost of updating OSMLR segments (and consider how to update any data associated to the segments) against the growing error rate in associating OSMLR back to OSM road segments.

Read more about considerations for the update/migration process, and track our progress, on this GitHub issue.

How to Download Preview OSMLR Segments Now

We have created a script that gives you the ability to fetch a subset of OSMLR GeoJSON or protocol buffer tiles.

This script is located within the py directory under the Open Traffic OSMLR repository. The script uses a bounding box to determine the list of tiles that intersect with the bounding box.

For example, if you would like to download OSMLR tiles for New York City in protocol buffer format you would execute the following command:

./download_tiles.sh -74.251961,40.512764,-73.755405,40.903125 https://s3.amazonaws.com/osmlr-tiles/v0.1/pbf ./tiles/ 4 osmlr false

And the GeoJSON tiles can be obtained via:

./download_tiles.sh -74.251961,40.512764,-73.755405,40.903125 https://s3.amazonaws.com/osmlr-tiles/v0.1/geojson ./tiles/ 4 json false

Note that this set of OSMLR segment tiles is considered a preview build. In the months ahead, we may make “breaking” changes to the ways in which OSMLR segments are generated and IDs are assigned.

So, please don’t yet depend on the stability of particular OSMLR segment IDs—but please do use this global tile set to begin exploring the possibilities for integrating OSMLR segments into your applications and analyses.

As we release new builds of OSMLR tile files, you’ll be able to view listing of the latest tile files on S3.

More in the Months Ahead

We’ll be following up in months ahead with another blog post to announce the OSMLR update/migration process. In the meantime, we hope you find this global set of OSMLR segment tiles interesting to explore and consider integrating into your own applications that need stable roadway identifiers.