Using Azure Timer Based Trigger to insert Geonames modifications into ArcGIS Online.

The Geonames database offers daily updates of their Gazetteer extract files. The record updates of the previous day are listed in a tab delimited file named “modifications-yyyy-MM-dd.txt”. I wanted to simple visualize the daily modifications in a point feature layer hosted at ArcGIS Online.

Geonames scheme

You have to create a feature layer representing the Geonames record scheme. The location of each record can be determined by using the latitude and the longitude field values. The 5th token represents the latitude and the 6th token represents the longitude. The first token is the unique ID of the Geonames record. The name, the country code and the last modification date are both added to the field scheme of the hosted feature layer.

Daily updates using timer based trigger

In Azure, you can implement a function which is going to be executed using a CRON expression. The function is using the default HTTP web client of the .NET Core Framework for accessing the daily modifications from the Geonames endpoint. The tab delimited file is read line by line using a memory stream. When implementing an Azure function you must not exceed 2 GB memory and the execution time must be less than 2 minutes.

Inserting records using the ArcGIS REST API

The Geonames records are converted into features. A feature is a record having field values, especially an integer based OBJECTID acting as the primary key and a shape field containing the geometry definition. The ArcGIS REST API allows edit operations which can be committed using a transaction. The feature service is secured and every client must be authorized before using any operations like query and edit. If you design a function which is not part of any user related workflow e.g. an app where a user is logged in using his/her credentials, you can register an app by using an appid and a clientid directly in ArcGIS Online. A registered app is allowed to access the available operations of the hosted feature services from the same organization. You can use the Anywhere.ArcGIS library developed by Dave Timmins to easily authorize your custom function with ArcGIS Online and inserting the Geonames records into a hosted feature service.

Visualizing the records using a simple web map

The web map is using a dark-colored basemap and the Geonames records are aggregated by their country code. The web map is hosted using ArcGIS Online and offers a user friendly map-based web-frontend.

Geonames_Overview
Geonames modifications aggregated using the country code
Azure-Geonames-Import
Geonames import used system components

 

Reference links: