Blog d’Anita Graser

https://anitagraser.com

  • Movement data in GIS #10: open tools for AIS tracks from MarineCadastre.gov 28 octobre 2017
    MarineCadastre.gov is a great source for AIS data along the US coast. Their data formats and tools though are less open. Luckily, GDAL – and therefore QGIS – can read ESRI File Geodatabases (.gdb). They also offer a Track Builder script that creates lines out of the broadcast points. (It can also join additional information from the vessel and voyage layers.) We could reproduce the line creation step using tools such as Processing’s Point to path. But this post will show how to create PostGIS trajectories instead. First, we have to import the points into PostGIS using either DB Manager or Processing’s Import into PostGIS tool: Then we can create the trajectories. I’ve opted to create a materialized view: The first part of the query creates a temporary table called ptm (short for PointM). This step adds time stamp information to each point. The second part of the query then aggregates these PointMs into trajectories of type LineStringM. CREATE MATERIALIZED VIEW ais.trajectory AS WITH ptm AS ( SELECT b.mmsi, st_makepointm( st_x(b.geom), st_y(b.geom), date_part(‘epoch’, b.basedatetime) ) AS pt, b.basedatetime t FROM ais.broadcast b ORDER BY mmsi, basedatetime ) SELECT row_number() OVE …
  • Movement data in GIS #9: trajectory data models 15 octobre 2017
    There are multiple ways to model trajectory data. This post takes a closer look at the OGC® Moving Features Encoding Extension: Simple Comma Separated Values (CSV). This standard has been published in 2015 but I haven’t been able to find any reviews of the standard (in a GIS context or anywhere else). The following analysis is based on the official OGC trajcectory example at http://docs.opengeospatial.org/is/14-084r2/14-084r2.html#42. The header consists of two lines: the first line provides some meta information while the second defines the CSV columns. The data model is segment based. That is, each line describes a trajectory segment with at least two coordinate pairs (or triplets for 3D trajectories). For each segment, there is a start and an end time which can be specified as absolute or relative (offset) values: @stboundedby,urn:x-ogc:def:crs:EPSG:6.6:4326,2D,50.23 9.23,50.31 9.27,2012-01-17T12:33:41Z,2012-01-17T12:37:00Z,sec @columns,mfidref,trajectory,state,xsd:token,”type code”,xsd:integer a, 10,150,11.0 2.0 12.0 3.0,walking,1 b, 10,190,10.0 2.0 11.0 3.0,walking,2 a,150,190,12.0 3.0 10.0 3.0,walking,2 c, 10,190,12.0 1.0 10.0 2.0 11.0 3.0,vehicle,1 Let’s look at the first da …
  • Movement data in GIS extra: trajectory generalization code and sample data 13 octobre 2017
    Today’s post is a follow-up of Movement data in GIS #3: visualizing massive trajectory datasets. In that post, I summarized a concept for trajectory generalization. Now, I have published the scripts and sample data in my QGIS-Processing-tools repository on Github. To add the trajectory generalization scripts to your Processing toolbox, you can use the Add scripts from files tool: It is worth noting, that Add scripts from files fails to correctly import potential help files for the scripts but that’s not an issue this time around, since I haven’t gotten around to actually write help files yet. The scripts are used in the following order: Extract characteristic trajectory points Group points in space Compute flows between cells from trajectories The sample project contains input data, as well as output layers of the individual tools. The only required input is a layer of trajectories, where trajectories have to be LINESTRINGM (note the M!) features: Trajectory sample based on data provided by the GeoLife project In Extract characteristic trajectory points, distance parameters are specified in meters, stop duration in seconds, and angles in degrees. The characteristic points contain s …
  • Movement data in GIS #8: edge bundling for flow maps 8 octobre 2017
    If you follow this blog, you’ll probably remember that I published a QGIS style for flow maps a while ago. The example showed domestic migration between the nine Austrian states, a rather small dataset. Even so, it required some manual tweaking to make the flow map readable. Even with only 72 edges, the map quickly gets messy: Raw migration flows between Austrian states, line width scaled by flow strength One popular approach in the data viz community to deal with this problem is edge bundling. The idea is to reduce visual clutter by generate bundles of similar edges.  Surprisingly, edge bundling is not available in desktop GIS. Existing implementations in the visual analytics field often run on GPUs because edge bundling is computationally expensive. Nonetheless, we have set out to implement force-directed edge bundling for the QGIS Processing toolbox [0]. The resulting scripts are available at https://github.com/dts-ait/qgis-edge-bundling. The main procedure consists of two tools: bundle edges and summarize. Bundle edges takes the raw straight lines, and incrementally adds intermediate nodes (called control points) and shifts them according to computed spring and electrostatic fo …
  • Drive-time Isochrones from a single Shapefile using QGIS, PostGIS, and Pgrouting 11 septembre 2017
    This is a guest post by Chris Kohler @Chriskohler8. Introduction: This guide provides step-by-step instructions to produce drive-time isochrones using a single vector shapefile. The method described here involves building a routing network using a single vector shapefile of your roads data within a Virtual Box. Furthermore, the network is built by creating start and end nodes (source and target nodes) on each road segment. We will use Postgresql, with PostGIS and Pgrouting extensions, as our database. Please consider this type of routing to be fair, regarding accuracy, as the routing algorithms are based off the nodes locations and not specific addresses. I am currently working on an improved workflow to have site address points serve as nodes to optimize results. One of the many benefits of this workflow is no financial cost to produce (outside collecting your roads data). I will provide instructions for creating, and using your virtual machine within this guide. Steps:–Getting Virtual Box(begin)– Intro 1. Download/Install Oracle VM(https://www.virtualbox.org/wiki/Downloads) Intro 2. Start the download/install OSGeo-Live 11(https://live.osgeo.org/en/overview/overview.html). Pictur …
  • Fixing invalid polygon geometries 29 août 2017
    Invalid geometries can cause a lot of headache: from missing features to odd analysis results. This post aims to illustrate one of the most common issues and presents an approach that can help with these errors. The dataset used for this example is the Alaska Shapefile from the QGIS sample data: This dataset has a couple of issues. One way to find out if a dataset contains errors is the Check Validity tool in the Processing toolbox: If there are errors, a layer called Error output will be loaded. In our case, there are multiple issues: If we try to use this dataset for spatial analysis, there will likely be errors. For example, using the Fixed distance buffer tool results in missing features: Note the errors in the Processing log message panel: Feature ### has invalid geometry. Skipping … So what can we do? In my experience, GRASS can work wonders for fixing these kind of issues. The idea is to run v.buffer.distance with the distance set to zero: This will import the dataset into GRASS and run the buffer algorithm without actually growing the polygons. Finally, it should export a fixed version of the geometries: A quick validity check with the Check validity tool confirms that th …
  • Getting started with GeoMesa using Geodocker 27 août 2017
    In a previous post, I showed how to use docker to run a single application (GeoServer) in a container and connect to it from your local QGIS install. Today’s post is about running a whole bunch of containers that interact with each other. More specifically, I’m using the images provided by Geodocker. The Geodocker repository provides a setup containing Accumulo, GeoMesa, and GeoServer. If you are not familiar with GeoMesa yet: GeoMesa is an open-source, distributed, spatio-temporal database built on a number of distributed cloud data storage systems … GeoMesa aims to provide as much of the spatial querying and data manipulation to Accumulo as PostGIS does to Postgres. The following sections show how to load data into GeoMesa, perform basic queries via command line, and finally publish data to GeoServer. The content is based largely on two GeoMesa tutorials: Geodocker: Bootstrapping GeoMesa Accumulo and Spark on AWS and Map-Reduce Ingest of GDELT, as well as Diethard Steiner’s post on Accumulo basics. The key difference is that this tutorial is written to be run locally (rather than on AWS or similar infrastructure) and that it spells out all user names and passwords preconfigured i …
  • Movement data in GIS #7: animated trajectories with TimeManager 14 août 2017
    In this post, we use TimeManager to visualize the position of a moving object over time along a trajectory. This is another example of what is possible thanks to QGIS’ geometry generator feature. The result can look like this: What makes this approach interesting is that the trajectory is stored in PostGIS as a LinestringM instead of storing individual trajectory points. So there is only one line feature loaded in QGIS: (In part 2 of this series, we already saw how a geometry generator can be used to visualize speed along a trajectory.) The layer is added to TimeManager using t_start and t_end attributes to define the trajectory’s temporal extent. TimeManager exposes an animation_datetime() function which returns the current animation timestamp, that is, the timestamp that is also displayed in the TimeManager dock, as well as on the map (if we don’t explicitly disable this option). Once TimeManager is set up, we can edit the line style to add a point marker to visualize the position of the moving object at the current animation timestamp. To do that, we interpolate the position along the trajectory segments. The first geometry generator expression splits the trajectory in its segme …
  • Dynamic styling expressions with aggregates & variables 22 juillet 2017
    In a recent post, we used aggregates for labeling purposes. This time, we will use them to create a dynamic data driven style, that is, a style that automatically adjusts to the minimum and maximum values of any numeric field … and that field will be specified in a variable! But let’s look at this step by step. (This example uses climate.shp from the QGIS sample dataset.) Here is a basic expression for data defined symbol color using a color ramp: Similarly, we can configure a data defined symbol size to create a style like this: Temperatures in July To stretch the color ramp from the attribute field’s minimum to maximum value, we can use aggregate functions: That’s nice but if we want to be able to quickly switch to a different attribute field, we now have two expressions (one for color and one for size) to change. This can get repetitive and can be the source of errors if we miss an expression and don’t update it correctly … To avoid these issues, we use a layer variable to store the name of the field that we want to use. Layer variables can be configured in layer properties: Then we adjust our expression to use the layer variable. Here is where it gets a bit tricky. We cannot si …
  • Docker basics with Geodocker GeoServer 12 juillet 2017
    Today’s post is mostly notes-to-self about using Docker. These steps were tested on a fresh Ubuntu 17.04 install. Install Docker as described in https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/ “Install using the repository” section. Then add the current user to the docker user group (otherwise, all docker commands have to be prefixed with sudo) $ sudo gpasswd -a $USER docker $ newgrp docker Test run the hello world image $ docker run hello-world For some more Docker basics, see https://github.com/docker/labs/blob/master/beginner/chapters/alpine.md. Pull Geodocker images, for example from https://quay.io/organization/geodocker $ docker pull quay.io/geodocker/base $ docker pull quay.io/geodocker/geoserver Get a list of pulled images $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE quay.io/geodocker/geoserver latest c60753e05956 8 months ago 904MB quay.io/geodocker/base latest 293209905a47 8 months ago 646MB Test run quay.io/geodocker/base $ docker run -it –rm quay.io/geodocker/base:latest java -version java version « 1.8.0_45 » Java(TM) SE Runtime Environment (build 1.8.0_45-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode) Run quay.io/geodoc …