You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Sarah Hoffmann fe5bd30147
Merge pull request #1023 from lonvia/gazetter-combine-deletes
19 hours ago
cmake Remove unused and superfluous stuff from CMake config. 3 weeks ago
contrib Merge pull request #1007 from lonvia/update-libosmium 1 week ago
docs Use single backtick in markdown files instead of two. 3 weeks ago
src Merge pull request #1023 from lonvia/gazetter-combine-deletes 19 hours ago
tests Switch functions in outputs vom int to void return types. 1 day ago
.clang-format persistent store: catch invalid location exception 2 years ago
.clang-tidy Add clang-tidy make target and basic clang-tidy config. 3 weeks ago
.gitignore Add tests/__pycache__ to .gitignore 1 week ago
.travis.yml Put fmt library into contrib and use it 1 week ago
AUTHORS Various small documentation changes. 3 weeks ago
CMakeLists.txt Put fmt library into contrib and use it 1 week ago
CONTRIBUTING.md Fix clang-format command line in CONTRIBUTING.md 2 weeks ago
COPYING * improved configure setup (including automake and libtool) 9 years ago
ChangeLog Revert "Remove the old SVN Changelog" 6 years ago
README Trim down README and move more detailed information into docs directory 5 years ago
README.md Include master branch name in travis badge 2 weeks ago
appveyor.yml Fix an issue with the platform env variable 1 year ago
default.style Small gramatic change 3 years ago
empty.style remove whitespace before EOL 4 years ago
install-postgis-osm-db.sh remove whitespace before EOL 4 years ago
install-postgis-osm-user.sh remove whitespace before EOL 4 years ago
multi.lua Correct spelling of superseded 3 years ago
multi.style.json remove comments in json configuration files 3 years ago
sample.multi.json fix tab in sample multi config 5 years ago
style.lua do not recheck for polygon tags in MPs 1 year ago

README.md

osm2pgsql

osm2pgsql is a tool for loading OpenStreetMap data into a PostgreSQL / PostGIS database suitable for applications like rendering into a map, geocoding with Nominatim, or general analysis.

Travis Build Status Appveyor Build status Packaging status

Features

  • Converts OSM files to a PostgreSQL DB
  • Conversion of tags to columns is configurable in the style file
  • Able to read .gz, .bz2, .pbf and .o5m files directly
  • Can apply diffs to keep the database up to date
  • Support the choice of output projection
  • Configurable table names
  • Gazetteer back-end for Nominatim
  • Support for hstore field type to store the complete set of tags in one database field if desired

Installing

Most Linux distributions include osm2pgsql. It is also available on macOS with Homebrew.

Unoffical builds for Windows are available from AppVeyor but you need to find the right build artifacts. Builds for releases may also be downloaded from the OpenStreetMap Dev server.

Building

The latest source code is available in the osm2pgsql git repository on GitHub and can be downloaded as follows:

$ git clone git://github.com/openstreetmap/osm2pgsql.git

Osm2pgsql uses the cross-platform CMake build system to configure and build itself and requires

Required libraries are

It also requires access to a database server running PostgreSQL 9.1+ and PostGIS 2.0+.

Make sure you have installed the development packages for the libraries mentioned in the requirements section and a C++ compiler which supports C++11. Both GCC 4.8 and Clang 3.4 meet this requirement.

First install the dependencies.

On a Debian or Ubuntu system, this can be done with:

sudo apt-get install make cmake g++ libboost-dev libboost-system-dev \
  libboost-filesystem-dev libexpat1-dev zlib1g-dev \
  libbz2-dev libpq-dev libproj-dev lua5.2 liblua5.2-dev

On a Fedora system, use

sudo dnf install cmake make gcc-c++ boost-devel expat-devel zlib-devel \
  bzip2-devel postgresql-devel proj-devel proj-epsg lua-devel

On RedHat / CentOS first run sudo yum install epel-release then install dependencies with:

sudo yum install cmake make gcc-c++ boost-devel expat-devel zlib-devel \
  bzip2-devel postgresql-devel proj-devel proj-epsg lua-devel

On a FreeBSD system, use

pkg install devel/cmake devel/boost-libs textproc/expat2 \
  databases/postgresql94-client graphics/proj lang/lua52

Once dependencies are installed, use CMake to build the Makefiles in a separate folder

mkdir build && cd build
cmake ..

If some installed dependencies are not found by CMake, more options may need to be set. Typically, setting CMAKE_PREFIX_PATH to a list of appropriate paths is sufficient.

When the Makefiles have been successfully built, compile with

make

The compiled files can be installed with

sudo make install

By default, the Release build with debug info is created and no tests are compiled. You can change that behavior by using additional options like following:

cmake .. -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTS=ON

Usage

Osm2pgsql has one program, the executable itself, which has 43 command line options.

Before loading into a database, the database must be created and the PostGIS and optional hstore extensions must be loaded. A full guide to PostgreSQL setup is beyond the scope of this readme, but with reasonably recent versions of PostgreSQL and PostGIS this can be done with

createdb gis
psql -d gis -c 'CREATE EXTENSION postgis; CREATE EXTENSION hstore;'

A basic invocation to load the data into the database gis for rendering would be

osm2pgsql --create --database gis data.osm.pbf

This will load the data from data.osm.pbf into the planet_osm_point, planet_osm_line, planet_osm_roads, and planet_osm_polygon tables.

When importing a large amount of data such as the complete planet, a typical command line would be

osm2pgsql -c -d gis --slim -C <cache size> \
  --flat-nodes <flat nodes> planet-latest.osm.pbf

where

  • <cache size> is about 75% of memory in MiB, to a maximum of about 30000. Additional RAM will not be used.
  • <flat nodes> is a location where a 36GiB+ file can be saved.

Many different data files (e.g., .pbf) can be found at planet.osm.org.

The databases from either of these commands can be used immediately by Mapnik for rendering maps with standard tools like renderd/mod_tile, TileMill, Nik4, among others. It can also be used for spatial analysis or shapefile exports.

Additional documentation is available on writing command lines.

Alternate backends

In addition to the standard pgsql backend designed for rendering there is also the gazetteer database for geocoding, principally with Nominatim, and the null backend for testing. For flexibility a new multi backend is also available which allows the configuration of custom PostgreSQL tables instead of those provided in the pgsql backend.

LuaJIT support

To speed up Lua tag transformations, LuaJIT can be optionally enabled on supported platforms. Performance measurements have shown about 25% runtime reduction for a planet import, with about 40% reduction on parsing time.

On a Debian or Ubuntu system, this can be done with:

sudo apt install libluajit-5.1-dev

Configuration parameter WITH_LUAJIT=ON needs to be added to enable LuaJIT. Otherwise make and installation steps are identical to the description above.

cmake -D WITH_LUAJIT=ON ..

Use osm2pgsql --version to verify that the build includes LuaJIT support:

./osm2pgsql --version
osm2pgsql version 0.96.0 (64 bit id space)

Compiled using the following library versions:
Libosmium 2.15.0
Lua 5.1.4 (LuaJIT 2.1.0-beta3)

Contributing

We welcome contributions to osm2pgsql. If you would like to report an issue, please use the issue tracker on GitHub.

More information can be found in CONTRIBUTING.md.

General queries can be sent to the tile-serving@ or dev@ mailing lists.