Blog d’Anita Graser

https://anitagraser.com

  • Movement data in GIS #14: updates from GI_Forum 2018 9 juillet 2018
    Last week, I traveled to Salzburg to attend the 30th AGIT conference and co-located English-speaking GI_Forum. Like in previous year, there were a lot of mobility and transportation research related presentations. Here are my personal highlights: This year’s keynotes touched on a wide range of issues, from Sandeep Singhal (Google Cloud Storage) who – when I asked about the big table queries he showed – stated that they are not using a spatial index but are rather brute-forcing their way through massive data sets, to Laxmi Ramasubramanian @nycplanner (Hunter College City University of New York) who cautioned against tech arrogance and tendency to ignore expertise from other fields such as urban planning: Next up: Sandeep Singhal Director, @Google Cloud Storage #GIForum2018 pic.twitter.com/fiRL9TUNpr — Anita Graser (@underdarkGIS) July 4, 2018 https://platform.twitter.com/widgets.js One issue that Laxmi particularly highlighted was the fact that many local communities are fighting excessive traffic caused by apps like Waze that suggest shortcuts through residential neighborhoods. Just because we can do something with (mobility) data, doesn’t necessarily mean that we should! L. Ramasu …
  • PyQGIS for non-programmers 6 juin 2018
    If you’re are following me on Twitter, you’ve certainly already read that I’m working on PyQGIS 101 a tutorial to help GIS users to get started with Python programming for QGIS. I’ve been toying around with the idea of a PyQGIS intro for non-programmers without all the boring parts of vanilla Python tutorials. Not sure how far this will go, but it’s a start: https://t.co/nbO5tm2ZW9 — Anita Graser (@underdarkGIS) May 31, 2018 https://platform.twitter.com/widgets.js I’ve often been asked to recommend Python tutorials for beginners and I’ve been surprised how difficult it can be to find an engaging tutorial for Python 3 that does not assume that the reader already knows all kinds of programming concepts. It’s been a while since I started programming, but I do teach QGIS and Python programming for QGIS to university students and therefore have some ideas of which concepts are challenging. Nonetheless, it’s well possible that I overlook something that is not self explanatory. If you’re using PyQGIS 101 and find that some points could use further explanations, please leave a comment on the corresponding page. PyQGIS 101 is a work in progress. I’d appreciate any feedback, particularly fro …
  • Scalable spatial vector data processing 18 mai 2018
    Working with movement data analysis, I’ve banged my head against performance issues every once in a while. For example, PostgreSQL – and therefore PostGIS – run queries in a single thread of execution. This is now changing, with more and more functionality being parallelized. PostgreSQL version 9.6 (released on 2016-09-29) included important steps towards parallelization, including parallel execution of sequential scans, joins and aggregates. Still, there is no parallel processing in PostGIS so far (but it is under development as described by Paul Ramsey in his posts “Parallel PostGIS II” and “PostGIS Scaling” from late 2017). At the FOSS4G2016 in Bonn, I had the pleasure to chat with Shoaib Burq who ran the “An intro to Apache PySpark for Big Data GeoAnalysis” workshop. Back home, I downloaded the workshop material and gave it a try but since I wanted a scalable system for storing, analyzing, and visualizing spatial data, it didn’t really seem to fit the bill. Around one year ago, my search grew more serious since we needed a solution that would support our research group’s new projects where we expected to work with billions of location records (timestamped points and associated …
  • Movement data in GIS #13: Timestamp labels for trajectories 9 mai 2018
    In Movement data in GIS #2: visualization I mentioned that it should be possible to label trajectory segments without having to break the original trajectory feature. While it’s not a straightforward process, it is indeed possible to create timestamp labels at desired intervals: The main point here is that we cannot use regular labels because there would be only one label for the whole trajectory feature. Instead, we are using a marker line with a font marker: By default, font markers only display one character from a given font but by using expressions we can make it display longer text, including datetime strings: If you want to have a label at every node of the trajectory, the expression looks like this: format_date( to_datetime(‘1970-01-01T00:00:00Z’)+to_interval( m(start_point(geometry_n( segments_to_lines( $geometry ), @geometry_part_num) ))||’ seconds’ ), ‘HH:mm:ss’ ) You probably remember those parts of the expression that extract the m value from previous posts. Note that – compared to 2016 – it is now necessary to add the segments_to_lines() function. The m value (which stores time as seconds since Unix epoch) is then converted to datetime and finally formatted to only sh …
  • Movement data in GIS #12: Why you should be using PostGIS trajectories 16 avril 2018
    In short: both writing trajectory queries as well as executing them is considerably faster using PostGIS trajectories (as LinestringM) rather than the commonly used point-based approach. Here are a couple of examples to give you an impression of the differences. Spoiler alert! Trajectory queries are up to 500 times faster than comparable point-based queries. A quick look at indexing In both cases, we have indexed the tracker id, geometry, and time columns to speed up query processing. The trajectory table has 3 indexes gist (time_range) gist (track gist_geometry_ops_nd) btree (tracker) The point-based table has 4 indexes gist (pt) btree (trajectory_id) btree (tracker) btree (t) Length First, let’s see how to determine trajectory length for all observed moving objects (identified by a tracker id). Using the point-based approach, we first need to ensure that the points are in the correct temporal order, create the lines, and finally sum up their length: WITH ordered AS ( SELECT trajectory_id, tracker, t, pt FROM geolife.trajectory_pt ORDER BY t ), tmp AS ( SELECT trajectory_id, tracker, st_makeline(pt) traj FROM ordered GROUP BY trajectory_id, tracker ) SELECT tracker, round(sum(ST_L …
  • Optional parameters in QGIS Processing scripts & models 1 avril 2018
    Remember the good old times when all parameters in Processing were mandatory? Inputs and outputs are fixed, and optional parameters or outputs are not supported. [Graser & Olaya, 2015] Since QGIS 2.14, this is no longer the case. Scripts, as well as models, can now have optional parameters. Here is how for QGIS 3: When defining a Processing script parameter, the parameter’s constructor takes a boolean flag indicating whether the parameter should be optional. It’s false by default: class qgis.core.QgsProcessingParameterNumber( name: str, description: str =  »,  type: QgsProcessingParameterNumber.Type = QgsProcessingParameterNumber.Integer,  defaultValue: Any = None,  optional: bool = False,  minValue: float = -DBL_MAX+1, maxValue: float = DBL_MAX) (Source: http://python.qgis.org/api/core/Processing/QgsProcessingParameterNumber.html) One standard tool that uses optional parameters is Add autoincremental field: From Python, this algorithm can be called with or without the optional parameters: When building a model, an optional input can be assigned to the optional parameter. To create an optional input, make sure to deactivate the mandatory checkbox at the bottom of the input parameter …
  • Processing script template for QGIS3 25 mars 2018
    Processing has been overhauled significantly for QGIS 3.0. Besides speed-ups, one of the most obvious changes is the way to write Processing scripts. Instead of the old Processing-specific syntax, Processing scripts for QGIS3 are purely pythonic implementations of QgsProcessingAlgorithm. Here’s a template that you can use to develop your own algorithms: from qgis.PyQt.QtCore import QCoreApplication, QVariant from qgis.core import (QgsField, QgsFeature, QgsFeatureSink, QgsFeatureRequest, QgsProcessing, QgsProcessingAlgorithm, QgsProcessingParameterFeatureSource, QgsProcessingParameterFeatureSink) class ExAlgo(QgsProcessingAlgorithm): INPUT = ‘INPUT’ OUTPUT = ‘OUTPUT’ def __init__(self): super().__init__() def name(self): return « exalgo » def tr(self, text): return QCoreApplication.translate(« exalgo », text) def displayName(self): return self.tr(« Example script ») def group(self): return self.tr(« Examples ») def groupId(self): return « examples » def shortHelpString(self): return self.tr(« Example script without logic ») def helpUrl(self): return « https://qgis.org » def createInstance(self): return type(self)() def initAlgorithm(self, config=None): self.addParameter(QgsProcessingParameterFeat …
  • Revisiting point & polygon joins 24 mars 2018
    Joining polygon attributes to points based on their location is a very common GIS task. In QGIS 2, QGIS’ own implementation of “Join attributes by location” was much slower than SAGA’s “Add polygon attributes to points”. Thus, installations without SAGA were out of good options. Luckily this issue (and many more) has been fixed by the rewrite of many geoprocessing algorithms for QGIS 3! Let’s revisit the comparison: I’m using publicly available datasets from Naturalearth: The small scale populated places (243 points) and the large scale countries (255 polygons with many nodes). Turns out that QGIS 3’s built-in tool takes a little less than two seconds while the SAGA Processing tool requires a litte less than six seconds: Like in the previous comparison, times were measured using the Python Console: In both tools, only the countries’ SOVEREIGNT attribute is joined to the point attribute table: import processing t0 = datetime.datetime.now() print(« QGIS Join attributes by location … ») processing.runAndLoadResults( « qgis:joinattributesbylocation », {‘INPUT’:’E:/Geodata/NaturalEarth/vector_v4/natural_earth_vector/110m_cultural/ne_110m_populated_places.shp’, ‘JOIN’:’E:/Geodata/NaturalEa …
  • Resources for QGIS3 21 février 2018
    The release of 3.0 is really close now. If you want to know what’s new or are just looking for interesting ways to pass the time until the packages land, check out the following QGIS3 resources. For users “24 Days of QGIS 3.0 Features” by Nyall Dawson “Exploring Reports in QGIS 3.0 – the Ultimate Guide!” by Nyall Dawson “Data exploration with Data Plotly for QGIS3” by your’s truly “Intro to QGIS3 3D view with Viennese building data” by your’s truly For more recordings from the developer meeting in Madeira check my Youtube playlist. For developers The official “Plugin migration to QGIS 3” wiki page “The PyQGIS Programmer’s Guide – Extending QGIS 3 with Python 3” by Gary Sherman “Porting Processing scripts to QGIS3” by your’s truly There’s still time to get the PyQGIS 3 Programmer’s Guide at discount before QGIS 3.0 is released! See: https://t.co/xJ7LLaX5UU #qgis #qgis3 #pyqgis #python pic.twitter.com/9k3FUmNFkH — Locate Press (@locatepress) February 13, 2018 Migrating your plugin to #qgis 3, you can use First Aid plugin to help you debugging issues. To learn more about First Aid:https://t.co/5k7RXUDfGD — Lutra Consulting (@lutraconsulting) February 1, 2018 If you have further readin …
  • TimeManager 2.5 published 12 février 2018
    TimeManager 2.5 is quite likely going to be the final TimeManager release for the QGIS 2 series. It comes with a couple of bug fixes and enhancements: Fixed #245: updated help.htm Fixed #240: now hiding unmanageable WFS layers Fixed #220: fixed issues with label size Fixed #194: now exposing additional functions: animation_time_frame_size, animation_time_frame_type, animation_start_datetime, animation_end_datetime Besides updating the help, I also decided to display it more prominently in the settings dialog (similarly to how the help is displayed in the field calculator or in Processing): So far, I haven’t started porting to QGIS 3 yet. If you are interested in TimeManager and want to help, please get in touch. On this note, let me leave you with a couple of animation inspirations from the Twitterverse: Here’s my attempt, Thanks @tjukanov for the inspiration and @dig_geo_com for the Tutorial. Distances traveled in 2,5,7,9,11,13,15 minutes in Cluj-Napoca, Romania. Made with @QGIS–#TimeManager,@openstreetmap, data from @here pic.twitter.com/tEhXfy6CAs — vincze istvan (@spincev) February 3, 2018 https://platform.twitter.com/widgets.js More typographic experiments with Danish ship dat …