{"id":5563,"date":"2018-07-02T05:32:13","date_gmt":"2018-07-02T03:32:13","guid":{"rendered":"http:\/\/www.sigterritoires.fr\/?p=5563"},"modified":"2018-10-17T13:01:41","modified_gmt":"2018-10-17T11:01:41","slug":"exploratory-analysis-of-data-for-geostatistics-the-qq-plot","status":"publish","type":"post","link":"https:\/\/www.sigterritoires.fr\/index.php\/en\/exploratory-analysis-of-data-for-geostatistics-the-qq-plot\/","title":{"rendered":"Exploratory analysis of data for geostatistics: the QQ-plot"},"content":{"rendered":"

Following the article\u00a0\u00a0\u00a0Introduction to exploratory data analysis for geostatistics<\/a>\u00a0\u00a0\u00a0we\u00a0\u00a0\u00a0will discuss each of the available tools to carry out the exploratory analysis of spatial data.\u00a0We have already discussed \u00a0 the histograms<\/a>\u00a0, and now we will address the QQ-Plots.<\/p>\n

QQ-Plots (or Quantile-Quantile Diagrams) are graphs in which the quantiles of two distributions are plotted against each other.<\/p>\n

Building a normal QQ-Plot<\/strong><\/p>\n

A\u00a0QQ-Normal Plot<\/em><\/strong>\u00a0is\u00a0the diagram that makes it possible to compare the distribution of the data of a batch with the so-called normal<\/em><\/strong>\u00a0\u00a0<\/em><\/strong>\u00a0or\u00a0\u00a0\u00a0Gaussian<\/em><\/strong>\u00a0distribution.\u00a0Here is an example.<\/p>\n

$\"\"$ <\/p>\n

How to build it?<\/p>\n

$\"\"$ <\/p>\n

AT –<\/strong>\u00a0\u00a0\u00a0The batch of data to be processed is ordered by value, from the smallest to the largest, and then the percentage of lower values \u200b\u200bis calculated for each value.\u00a0We plot the values \u200b\u200bof the batch on the abscissa and the percentages on the ordinates.\u00a0In this example on the ordinates for the value 2 corresponds 21% (0,21) of lower values \u200b\u200bpresent in the batch (and thus 79% of values \u200b\u200bgreater than 2).<\/p>\n

B-<\/strong>\u00a0\u00a0<\/strong>\u00a0The Gaussian function is plotted with the standard deviations on the abscissa and the frequency percentage inferior to this value on the ordinates. For a frequency equal to 21% (0.21), the standard deviation equals -0.85.<\/p>\n

C-<\/strong>\u00a0We create the QQ-Plot:<\/p>\n

\n
we use the value (DV) for each data,<\/li>\n
we look for the percentage of graph A,<\/li>\n
with this percentage, we move to\u00a0\u00a0\u00a0graph B and obtain the corresponding standard deviation value (NV),<\/li>\n
we draw the point using NV on the abscissa and DV on the ordinates.<\/li>\n<\/ul>\n
The right portion of the QQ-Plot indicates the position that should have the points if they matched exactly the normal distribution.<\/p>\n
How to build a general QQPlot<\/strong><\/p>\n
The\u00a0\u00a0\u00a0QQ-General Plot<\/em><\/strong>\u00a0\u00a0\u00a0is used to evaluate the similarity between the distributions of two sets of data.<\/p>\n
$\"\"$ <\/p>\n
Here we have two variables: Depth and Distance<\/p>\n
How to build it?<\/p>\n
$\"\"$ <\/p>\n
A –<\/strong>\u00a0\u00a0\u00a0As for the normal QQ-Plot, the first\u00a0\u00a0\u00a0batch of data to be processed is ordered by value, from the smallest to the largest, then the percentage of lower values \u200b\u200bis calculated for each value.\u00a0We plot on the ordinates the values \u200b\u200bof the lot and on the abscissa the percentages.\u00a0In this example for the value 2 in our data we have 21% (0,21) of lower values \u200b\u200bpresent in the batch.<\/p>\n
B –<\/strong>\u00a0\u00a0\u00a0The second batch of data is treated in the same way.\u00a0In this example for the value in our data we have 37% (0.37) of lower values \u200b\u200bpresent in the batch.\u00a0You will notice that there is no value in the batch with a frequency of 0.21 as in the first batch of data.<\/p>\n
C-<\/strong>\u00a0We create the QQ-Plot:<\/p>\n
\n
for each data of lot A we select its value (DV1),<\/li>\n
we look for the percentage of graph A,<\/li>\n
with this percentage, we move to\u00a0\u00a0\u00a0graph B and obtain the value of the corresponding batch B (DV2), either by selecting it directly (whenever possible), or by interpolating between the two encompassing values \u200b\u200b, as in the example above.<\/li>\n
we draw the point using DV2 on the abscissa and DV1 on the ordinate,<\/li>\n
for each data of the batch B we select its value (DV2),<\/li>\n
we look for the percentage of chart B,<\/li>\n
using this percentage, we move to\u00a0\u00a0\u00a0graph A and obtain the value of the corresponding batch A (DV1), either by selecting it directly (whenever possible), or by interpolating between the two values \u200b\u200bwhich encompass it, as in the example above.<\/li>\n
we draw the point using DV2 on the abscissa and DV1 on the ordinate.<\/li>\n<\/ul>\n
Unlike the normal QQ-Plot, we can not draw theoretical line because we do not know the distribution function of the lots A and B. However, if the two distributions are exactly the same, the points will be aligned on a straight line.\u00a0In the above example (Depth-Distance) this is not the case.
\nThe following example is a perfect match (since it’s the same variable):<\/p>\n
$\"\"$ <\/p>\n
Interpretation of QQ-Plots<\/strong><\/p>\n
We will repeat here what we have, already, said for the histograms:<\/p>\n
\u00ab\u00a0<\/em>\u00a0\u00a0<\/em>\u00a0Some kriging methods work best if the data is distributed approximately normally (the bell-shaped curve).<\/em>
\nIn particular, quantile and probability maps using ordinary, simple and universal kriging assume that the data come from a normal distribution.<\/em>
\nAs we saw in the previous article, kriging is also based on the hypothesis of stationarity.<\/em>\u00a0\u00a0<\/em>\u00a0This assumption requires, in part, that all data values <\/em>\u200b\u200b<\/em>come from distributions that have the same variability. In nature, we often observe that as values <\/em>\u200b\u200b<\/em>increase, their variability, also, increases.<\/em>\u00a0\u00a0<\/em>\u00a0The<\/em>\u00a0\u00a0<\/em>\u00a0transformations<\/em><\/strong>\u00a0of\u00a0source data can be used to transform your data into a normal \u00a0distribution and satisfy the assumption of equal variability for the whole set.<\/em>\u00a0\u00ab\u00a0<\/em>\u00a0\u00a0<\/em><\/p>\n
Therefore, we will look for the same things as with the histograms, but with the QQ-Plot it will be easier.<\/p>\n
If we select the variable\u00a0\u00a0\u00a0Depth<\/em><\/strong>, used for the histogram and we draw its normal QQ-Plot we have:<\/p>\n
$\"\"$ <\/p>\n
We have three different areas:<\/p>\n
\n
A- Points to the left of the theoretical line, very far from this one<\/li>\n
B- Points to the right of the theoretical line, and<\/li>\n
C- Points to the left, again<\/li>\n<\/ul>\n
The general shape is equivalent to an S.<\/p>\n
The information that we can draw from the general shape of the points curve is mainly related to the form coefficients: skewness and kurtosis.\u00a0Moreover, we can immediately observe if our data follow a mono or bi-modal curve.<\/p>\n
Observation of the spreading<\/strong><\/p>\n
First, a few words on spreading (\u00a0skewness<\/em><\/strong>\u00a0).<\/p>\n
We have three main types of distribution: normal, moved to the left (towards the small values \u200b\u200bof our data), moved to the right (towards the big values \u200b\u200bof our data).<\/p>\n
In order to, quickly, find which the type of our distribution is; look at the corresponding QQ-Plot area in the centre of our distribution (Value 0 of the standard deviation):<\/p>\n
UNBIASED DISTRIBUTION (NORMAL):\u00a0<\/strong><\/p>\n
$\"\"$ <\/p>\n
The points of the data corresponding to the centre of the distribution are included (or very close) in the theoretical line.<\/p>\n
DISTRIBUTION BIASED to the LEFT:\u00a0<\/strong><\/p>\n
$\"\"$ <\/p>\n
The area of \u200b\u200bpoints around 0 standard deviation is substantially below the theoretical line.<\/p>\n
\u00a0DISTRIBUTION BIASED TO THE RIGHT:\u00a0<\/strong><\/p>\n
$\"\"$ <\/p>\n
The area of \u200b\u200bpoints around 0 standard deviation is substantially above the theoretical line.<\/p>\n
Observation of the flattening<\/strong><\/p>\n
The other possible observation concerns the coefficient of spread (kurtosis).<\/p>\n
KURTOSIS LESS THAN 3<\/strong><\/p>\n
Distributions with relatively thin edges (called platykurtic) and which have a kurtosis\u00a0value lower than 3, have a general S shape, with the negative part of the standard deviations concave, and the positive part convex:<\/p>\n
$\"\"$ <\/p>\n
KURTOSIS GREATER THAN 3<\/strong><\/p>\n
Distributions with relatively thick edges (called leptokurtic)\u00a0\u00a0\u00a0and which have a kurtosis \u00a0value greater than 3, have a general inverted S shape, with the negative section of the standard deviations convex, and the positive section concave:<\/p>\n
$\"\"$ <\/p>\n
What to do?<\/strong><\/p>\n
Using the normal QQ-plot there are two things we can do: find a transformation that brings our data back to a normal (or near) distribution and identify the data that can be problematic.<\/p>\n
If we consider the first diagram of this article, it is easier to find the exponential transformation (Box-Cox) with the QQ-Plot than with the histogram:<\/p>\n
$\"\"$ <\/p>\n
When we modify the transformation parameter, we get a better picture of the adequacy to the theoretical line..<\/p>\n
The other interesting aspect of the Geostatistical Analyst is the link between the diagram tools, here the QQ-plot and the display in ArcMap.\u00a0If you use the selection tool on QQ-Plot points you will observe the selected points on the map.<\/p>\n
If you select points that deviate from the normal line for large values:<\/p>\n
$\"\"$ <\/p>\n
We realize that they are all in the periphery of the area considered.\u00a0They can therefore translate an external phenomenon to our zone.\u00a0This will be kept in mind, for example, to test the quality of the final interpolation with or without these points.<\/p>\n
If we do the same thing for points deviating in small values: figura 14<\/p>\n
We observe that the distribution of these points belong to the phenomenon intrinsic to the area considered, or that, in any case, it will be necessary to take them into account for the modelling of the interpolation function.<\/p>\n
In the next article we will see how to detect outliers with Voronoi polygons.<\/p>\n","protected":false},"excerpt":{"rendered":"
Following the article\u00a0\u00a0\u00a0Introduction to exploratory data analysis for geostatistics\u00a0\u00a0\u00a0we\u00a0\u00a0\u00a0will discuss each of the available tools to carry out the exploratory analysis of spatial data.\u00a0We have already discussed \u00a0the histograms\u00a0, and now we will address the QQ-Plots.…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"give_campaign_id":0,"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"sfsi_plus_gutenberg_text_before_share":"","sfsi_plus_gutenberg_show_text_before_share":"","sfsi_plus_gutenberg_icon_type":"","sfsi_plus_gutenberg_icon_alignemt":"","sfsi_plus_gutenburg_max_per_row":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1260],"tags":[],"class_list":["post-5563","post","type-post","status-publish","format-standard","hentry","category-non-classe-en"],"aioseo_notices":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p6XU0A-1rJ","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.sigterritoires.fr\/index.php\/wp-json\/wp\/v2\/posts\/5563","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.sigterritoires.fr\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.sigterritoires.fr\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.sigterritoires.fr\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.sigterritoires.fr\/index.php\/wp-json\/wp\/v2\/comments?post=5563"}],"version-history":[{"count":0,"href":"https:\/\/www.sigterritoires.fr\/index.php\/wp-json\/wp\/v2\/posts\/5563\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.sigterritoires.fr\/index.php\/wp-json\/wp\/v2\/media?parent=5563"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.sigterritoires.fr\/index.php\/wp-json\/wp\/v2\/categories?post=5563"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.sigterritoires.fr\/index.php\/wp-json\/wp\/v2\/tags?post=5563"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}