Spatial regression graph convolutional neural networks: A deep learning paradigm for spatial multivariate distributions

Abstract

Geospatial artificial intelligence (GeoAI) has emerged as a subfield of GIScience that uses artificial intelligence approaches and machine learning techniques for geographic knowledge discovery. The non-regularity of data structures has recently led to different variants of graph neural networks in the field of computer science, with graph convolutional neural networks being one of the most prominent that operate on non-euclidean structured data where the numbers of nodes connections vary and the nodes are unordered. These networks use graph convolution – commonly known as filters or kernels – in place of general matrix multiplication in at least one of their layers. This paper suggests spatial regression graph convolutional neural networks (SRGCNNs) as a deep learning paradigm that is capable of handling a wide range of geographical tasks where multivariate spatial data needs modeling and prediction. The feasibility of SRGCNNs lies in the feature propagation mechanisms, the spatial locality nature, and a semi-supervised training strategy. In the experiments, this paper demonstrates the operation of SRGCNNs with social media check-in data in Beijing and house price data in San Diego. The results indicate that a well-trained SRGCNN model is capable of learning from samples and performing reasonable predictions for unobserved locations. The paper also presents the effectiveness of incorporating the idea of geographically weighted regression for handling heterogeneity between locations in the model approach. Compared to conventional spatial regression approaches, SRGCNN-based models tend to generate much more accurate and stable results, especially when the sampling ratio is low. This study offers to bridge the methodological gap between graph deep learning and spatial regression analytics. The proposed idea serves as an example to illustrate how spatial analytics can be combined with state-of-the-art deep learning models, and to enlighten future research at the front of GeoAI.

Appendices

Appendix

In this section, we discuss SRGCNNs in a more typical spatial regression scenario: house price modeling. Utilizing a house rent price dataset at San Diego, C.A., U.S., we carefully evaluate the regression accuracies across models, and investigate model performances given different sets of explanatory variables.

A.1 Data description and feature selection

The open data behind the Inside Airbnb site³ was collected for this appendix experiment. The data is sourced from publicly available information in the Airbnb site, which includes the daily rent price of listed properties and many additional attributes for the listings. To compare SRGCNNs with traditional spatial regression models, we will examine some information in all Airbnb listings in San Diego, C.A., U.S. on July, 07, 2016.

In Fig. 11, we visualize the logarithm prices to the base e (ln(price)) in Fig. 11a. The map utilizes a percentile color scheme to highlight both the extreme high prices (head 1%, in yellow) and low prices (tail 1%, in black). There are 6,110 collected Airbnb listings in total, the prices are obviously spatial autocorrelated, with high-value clusters as well as low-value clusters within the study area. In Fig.11b, we present the house characteristics that are of interest in this experiment, with both continuous variables (e.g., number of accommodate people as in “accommodates”, and number of beds as in “beds”) and categorical variables (e.g., rent type as in “rt_XXX”, and property groupy in “pg_XXX”). Also, we include a binary variable named “coastal” to indicate whether a house is near the ocean.

Fig. 11

Daily price in local currency for Airbnb listings in San Diego, U.S., collected on July, 07, 2016. a Map visualization of ln(price). b Listing’s attributes as the interested variables

In the following regression analysis, we will use ln(price) as the dependent variable y, and examine two independent sets X (four variables) and X₊ (eleven variables). The basic independent variable set X = {“accommodates”, “bathrooms”, “bedrooms”, “beds”} contains only the four continuous intrinsic characteristics: number of accommodate people, number of bedrooms, number of bathrooms, and number of beds. While the extended independent variable set X₊ = {“accommodates”, “bathrooms”, “bedrooms”, “beds”, “rt_Private_room”, “rt_Shared_room”, “pg_House”, “pg_Condominium”, “pg_Townhouse”, “pg_Other”, “coastal”} contains additional characteristics of rent type, property group, and the coastal indicator. The rent types are used as dummy variables, denoting whether a listing belongs to a private room, a shared room, or an entire home. The property groups are also used as dummy variables, indicating whether the listing is an apartment, a condom, a townhouse, a single family house or others.

Note that it is possible to include other surrounding environmental context as the independent variables, such as the distance to the highways and number of parks in the neighborhood to further improve the regression accuracy. However, the selection of informative feature variables is beyond the scope of our paper. Here, we just provide two different sets of independent variables in order to shed light on the influence of feature engineering in SRGCNN-based models. Future applications are invited to test out SRGCNNs with different feature combinations in specialized tasks.

A.2 Model training

The regressions on prices at all locations (100% training ratio) are performed using linear regression model (LR), spatial autoregressive model (SAR), and the SRGCNN-GW model (19). For simplicity, models with the additional variable set X₊ are referred to as LR+, SAR+, and SRGCNN-GW+, respectively. We choose SRGCNN-GW model here rather than the basic SRGCNN model because SRGCNN-GW is better at fitting the training dataset, while the basic SRGCNN model is better for prediction (as discussed in Section 5.2).

We consider k= 20 nearest neighbors for each location to construct the spatial weights matrix in SAR and the graph structure in SRGCNN-GW. It is optional to change the way of defining the spatial structure, e.g., a different k, or using other measurements such as distance, queen adjacency. We won’t dive into this because the influence of geographic contexts on spatial regression is another topic to investigate [25] and it is beyond the scope of this paper.

We adopt similar training settings as introduced in Section 4.2.3. The learning rate is changed to η = 3 × 10^− 2. Training epochs are capped at 15,000 for SRGCNN-GW and 18,000 for SRGCNN-GW+. We record the best results among all epochs. The MSE Loss and MAPE during the training process are plotted in Fig. 12. The hidden feature units are set to be 4 × 8 = 32 for SRGCNN-GW and 11 × 8 = 88 for SRGCNN-GW+, considering the different input features provided in X and X₊. As can be seen, SRGCNN-GW reaches the lowest MAPE at epoch 12,161, while SRGCNN-GW+ reaches its lowest MAPE at epoch 15,508. After that, both models exhibit overfitting as the MAPE starts to rise up again. For the whole training, SRGCNN-GW+ converged slightly slower (with more epochs) compared to SRGCNN-GW, because there are more feature parameters to be learned in the geographic weighted graph convolutions.

Fig. 12

Training process of the SRGCNN-GW models (a) SRGCNN-GW: X as the explanatory variables. b SRGCNN-GW+: X₊ as the explanatory variables

A.3 Evaluation of the results

The results across all models are summarized in Table 4. We report the R² and MAPE as two metrics to evaluate the goodness of model fitting. Also, the Z-scored Morans’ I value of prediction errors and the fitted ln(price) are included to indicate how the models capture the spatial effects in data.

Table 4 Model performances on fitting all prices (sampling ratio:100%)

It is encouraging to find that SRGCNN-GW significantly outperforms LR and SAR regarding both R² and MAPE. Using the basic independent variable set X, we can see that LR, the non-spatial linear regression model, can only explain about 56% of the real process; SAR, the most common used spatial lagged model, increases the goodness of fit to about 62%; SRGCNN-GW model, however, reaches a much higher goodness of fit around 83%. By adding more explanatory information, all models exhibit better results using X₊. LR increases from 55% to 68%, SAR increases from 62% to 71%, and SRGCNN increases from 83% to 87%. Since SRGCNN-based models consider more complex spatial relationships during the modeling, the influence of additional independent variables is less than traditional models such as linear regression and spatial lagged models. With respect to the MAPE, conclusions are exactly the same, SRGCNN-GW+ reaches the lowest fitting error at only 3.76%. Seen from the Z-scored autocorrelations, SAR and SAR+ are better at handling the spatial errors. SRGCNN-GW models also have lower error autocorrelations compared to LR models. SRGCNN-GW models reports higher global autocorrelation of the fitted price than SAR, indicating an explicitly modeling of spatial structure in its graph convolution layers.

Results are also compared in Fig. 13. The spatial distributions of ln(price) are plotted in the first row for both the original data and the model predictions. The scatter plots and the Pearson correlation coefficients ρ are presented in the second row to further evaluate the models. As shown, SRGCNN-GW+ has done an outstanding job fitting the price data, with a modeled spatial pattern really similar to the original one and a highest Pearson correlation ρ = 0.9334. The number of input features does have influences on the modeling accuracies, but it is still not clear on how to select informative variables for SRGCNN models. Future works are to develop specialized methods for the visualization and analysis of complex feature parameters in SRGCNN models with regard to regression statistics.

Fig. 13

Model comparison in maps and scatterplots