Setting up your own OpenStreetMap Server

Hi Guys,

I will try to keep it simple and straight forward. Most of the content has been taken from openstreetmap.org‘s wiki.

I end up configuring it last night so it is still fresh in my brain.

Alright first things first

If you are thinking for running a OSM server on a machine that has 4GB RAM and Intel Core Dual Core Processor then forget about it. You will need really good hardware to run your OSM server. I recommend at least 24 GB RAM and a Xeon quad core processor (Dual) with probably super fast iSCSI drives or good SSD.

To import full planet OSM data you will need more than 600GB of free Hard disk space. It can take up to 10+ days depending on your Hardware to import and Index everything planel.osm has to offer.

I will be using Ubuntu Desktop version 11.10 64 bit. for this example.

So prepare your new machine with basic Ubuntu Desktop version 11.10 64 bit. install. No need to install anything else. You can choose to download updates for the OS.

After you can login to your newly laid Ubuntu Desktop. Do the Following

  1. Download latest planet.osm data or you can try to search for extracts for your Country if you don’t want to import full size planet data. I will be using Australia.osm. Thanks to Matt who is running this really helpful website http://www.osmaustralia.org
  2. (optional) Extract the bz2 file to an accessible location on your Hard drive.

So now we have good Hardware and we’ve got the required OSM data files its time to get our hands dirty.

Open your Ubuntu Terminal

* TIP nearly all command will require to be run as Super User i.e. administrator.

I will be working in this directory /home/osm. You can will need to choose your own whenever possible.

Lets set up the environment first. As a prerequisites we need to install following packages

  • GCC compiler
  • PostgreSQL
  • Proj4
  • GEOS
  • PostGIS
  • PHP
  • PHP-pgsql
  • PEAR::DB

prompt>  sudo apt-get install php5-pgsql postgis postgresql php5 php-pear gcc g++ php-pear gcc proj libgeos-c1 postgresql-contrib postgresql-9.1-postgis postgresql-server-dev-9.1 Lets install DB package 

Now that we’ve got postgreSQL installed we will tune its performance params to suit OSM importing

As per the parent article do this. You can find these params in  file

You should tune the following parameters:

  • shared_buffers = 4GB
  • maintenance_work_mem = 4GB
  • work_mem =10MB
  • synchronous_commit = off
  • full_page_writes = off
  • checkpoint_segments = 50
  • checkpoint_timeout = 10min
  • checkpoint_completion_target = 0.9
  • random_page_cost = 1.5 #(lower than the default of 4).

The numbers in brackets behind some parameters seem to work fine for 32GB RAM machine. You can adjust these as per your machine hardware. You can leave the default values too but that will means that you wait longer for OSM data to get imported to your DB.

After saving the file, You must restart postgreSQL server

For the initial import, you should also set:

  • fsync = off
  • autovacuum = off

Lets install postgreSQL client


Lets install git


Lets install subversion (only required if you want to build osm2pgsql util from source)


We will need osm2pgsql to import our OSM data to postgreSQL (well not directly but this will be used by Nominatim).

Important

 

We have two choices,

  1. build osm2pgsql from source
  2. or just install the available package

I will recommend 2 but Lets cover 1 too.

To build osm2pgsql from source run the following commands

Or do this (easy option)

Now that we have osm2pgsql installed its time to get Nominatim

Nominatim  is a tool to search osm data by name and address and to generate synthetic addresses of osm points (reverse geocoding). It on the other hand has a website interface too.

Lets install Nominatim from source

First lets checkout/clone Nominatim from GIT

Now lets build it.

Lets create a website user

Please note that You also need a user with supervisor rights for the account that is doing the import.

Create the Role if it does not exist. I had fun doing that but its straight forward. If you will be running this server under your current user your then create a postgreSQL role with current user name.

The stage is set to import the data.

Make sure that under your /path/to/Nominatim/settings/local.php path to osm2pgsql and CONST_Website_BaseURL constant are right. Audit other constants and change as per required.

Run the command below to import OSM data to your postgreSQL server.

You will know in 5-10 minutes if there is something wrong. If you go pass that point leave your machine ON for many days (roughly 10+) to come.

If you get errors like database nominatim already exists then drop that database and try again until you get all the requirements sorted.

Once your data is in, Do this

Lets add country codes and names to search index

It is also worthwhile to import special phrases to your database. Commands are shown below

special phrases are deemed a good Idea when you provide search for special amenities. Say for example if you are searching for hotels in Carlton Victoria will search index to find hotels in Carlton

Final thing is to setup a website so that you can make your OSM data public

Run the command below to setup a Nominatim website

If the above step fails then check your directory permissions and make sure that Write permission is given. You can create nominatim directory under /var/www as shown below and then run the command above

Under your /etc/apache2/httpd.conf file add these lines

If you want your default website to be your OSM maps then you will need to edit /etc/apache2/site-available/default and change /var/www to /var/www/nominatim

Thats it

restart your apache server /etc/init.d/apache restart and address your browser to whatever domain you’ve mentioned in settings/local.php under this contant  CONST_Website_BaseURL

You will see a page like this.

Now when I try to search for bundoora victoria, I get the following result

 

 

I hope this helps

Cheers!

Comments

  1. thank you man for this tuto, very useful…but i’m blocked now in the step of import the data. i created a database and name it nominatim and i tried this command: ‘sudo ./utils/setup.php \
    –osm-file morocco-latest.osm.pbf
    –all –osm2pgsql-cache 18000 2>&1 \
    | tee setup.log’ it shows me this message :
    ‘CREATE DB
    ERROR: database already exists ( pgsql : // @/nominatim) ‘
    Now hen i delete my database nominatim it shows this error message : ‘ERROR: unable to find /usr/pgsql-9.3/share/contrib/postgis-1.5/postgis.sql’ btw i installed postgres with all the package. Any idea ??

  2. Thank you very much for this great tut,, I am able to get my local tile server displayed. But keen to know is there a way to upload my own .GPX track records to local tile server or make tiles with .gpx and upload.
    your help is very much appreciated, Thank you again.

  3. Thanks for your Detail installation Procedure. I am able to install the same.After installation, It display map without problem. But it gives a error on top

    “Data: DB Error: insufficient permissions ”

    Please advise.

    • Make sure that your DB user have escalated privileges such as delete, update, create etc I haven’t seen that one before but looking into user privs looks logical here.
      cheers,

  4. Hi,

    I am trying to find a 64bit version of osm2pgsql windows executable. I tried the 32bit one with the 64bit ids enabled and it wont work, any suggestions?

  5. Hi guys,
    I followed this tuto http://wiki.openstreetmap.org/wiki/Nominatim/Installation.

    Like Jaspreet Chahal, the map is getting tiles from OSM server and not mine. I tried changing http://b.tile.openstreetmap.org/${z}/${x}/${y}.png by http://localhost/nominatim/${z}/${x}/${y}.png in tiles.js but it didn’t work. Indeed, there is no tiles in localhost/nominatim.

    Where are located the tiles?

    Thanks guys
    Lucas

  6. Thanks for your guide 🙂
    I have one “litte” problem after setting up my server. The setup with the Nominatim scrip runs without errors, but wenn I access the servers webinterface my browser tries to load the OSM tiles from the internet and not from my local server.
    The following is from the Firefox Web-Console:
    GET http://b.tile.openstreetmap.org/4/9/8.png [HTTP/1.1 200 OK 175ms]
    GET http://c.tile.openstreetmap.org/4/10/8.png [HTTP/1.1 200 OK 144ms]
    GET http://c.tile.openstreetmap.org/4/11/8.png [HTTP/1.1 200 OK 145ms]
    GET http://a.tile.openstreetmap.org/4/11/7.png [HTTP/1.1 200 OK 155ms]

    Do you have any idea why?

    • That’s the way it should be. Right? If I am not wrong then the download size of your osm file will be many more gigs and storage requirement will triple. I have to look into this a bit more. I can’t see any different download for osm imagery. I think they would like to keep that data in central servers. Maybe it’s worth asking on their forums.
      Cheers

      • Why should this be the way it should be? From my point of view it doesn’t make sense. Why should I set up my own OSM server, render all the data and then the whole data came from the internet?
        My motivation to set up my own OSM-Server is to use it in a local intranet where is no connection to the internet, so I need the data local.
        Asking in their forums could be a try.

  7. Luis Moscoso says

    Hi, why reversegeocoding response are diferents betweeen openstreet server and my server?

    hoy can i get the same data or openstreet or update

    • Hi Luis,
      Just check if your planet.osm (or your_country.osm) import is up2date. I personally never had this problem. As you may be aware that OSM is literally updated so very frequently. So keeping an up2date data is important on your local server. Also check your queries if they are consistent and you are doing them right.
      Cheers!

  8. Hi!

    Thank you for this tutorial! It is great!
    I succeeded to import the OSM data for Netherlands (by the way, took 38 hours to finish) and now I want also to import (in the same database) the OSM data for Belgium and Luxembourg.
    Do you know how can be done?
    I have tried using ./utils/setup.php –osm-file belgium.pbf –import-data , but didn’t work (stopped at some point: Setting up table: planet_osm_ways)

    Best regards,
    Roxana

    • Hi Roxana,
      I personally never tried that because I sort of limit my Imports to Australia OSM only. I think that importing multiple OSMs can not be done yet but I could be wrong. I will try to find an answer though. For cases like yours I would just import Planet OSM (I know the import time are really lengthy thats why having a good hardware machine is so important)
      Cheers!

  9. Hi,

    Thanks for your tutorial.
    I still not finish to execute it, but one thing disturb me: how the tiles are generated ? I read two times the page and didn’t understand how and when you generate the tiles, like others do with Mapnik.

    Regards.
    Jean-Luc

  10. Hi,’
    Thanks for the great info. I would like to ask what are the specs for the server/storage device you are using/recommending for OSM install

    • I am using an i7 processor with 10k RPM SATA disks, A combined space of over 2 TB. RAM for my machine is 32 GB.
      Thing here is that the more room and processing power you give to your OSM import the faster it will be. I consider my configuration as serious for my standards.
      I would recommend that if you got RAM more than 32GB and Disk I/O to 10K (2 TB is good number for future expansions I guess) with a decent Xeon processor that will make your OSM imports quicker (it depends what you are trying to do, for a one off home project your can use high end AWS too for all this stuff rather than spending $$s on brand new hardware)
      A simple Quad core machine with 16GB RAM and a good high end Disks should be relatively slower (take days) but will be ok for all import in thin mode. The only thing that add to frustration is when import sort of fails half way through because of any given reason. So a better hardware backed up with an UPS seems to be a good idea

      Cheers!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.