Forecasting Housing Prices with HPI

Research and analysis of housing market behaviors has gained additional significance in recent years, as several analysts have been attributing major importance to the housing price dynamics and their impact on the economy. Various attempts have been made to investigate and explain these dynamics and the characteristics of the housing market.

During several presentations, we too have been asked about our opinion on using housing price indexes to forecast housing price movements and our ability to incorporate economic data within our data abstraction model. We decided to take on this challenge and contribute to the debate. We easily added the housing price data into our data model. Then we integrated R statistics (The R Project for Statistical Computing) into the domain model to facilitate statistical analysis of housing price data.

In particular, we tried to understand if some kind of understandable trend and, consequently, actual possibilities of forecasting exist. Our study developed through a series of statistical tests on the FHFA House Price Indexes (HPI) for the 100 Largest Metropolitan Statistical Areas and Divisions (MSAD) in the United States, from 1991 through the second quarter of 2013. At several points of the analysis, we also used the Standard & Poors (SP) Case-Shiller Index to confirm our results.
Eventually, but somehow predictably, what started as an attempt of forecasting future prices using existing models turned into something different. Basically, the conclusions we have come to are the following:

  • The data does not allow for a single parsimonious model to analyze and forecast housing prices, using either FHFA or SP Case-Schiller HPI data.
  • The stochastic process for housing prices differs significantly at the MSAD level. Specifically, we found that of the largest 50 MSAD, only 21 are stationary under the first difference (integrated of order one). For the remaining 29 MSAD, we had to run further tests and received contradictory results from the different stationarity and unit root tests.
  • We are not able to reject the null hypotheses of the presence of a unit root using both the Phillips-Perron and Schmidt-Phillips tests for none of the considered MSAD.
  • Given these substantial differences in the stochastic processes for the different MSAD, it would be meaningless to perform an aggregate analysis at the national level.

Our research began with a basic autoregressive model with OLS (Ordinary Least Squares). We applied the model to the HPI for all 100 MSAD, analyzing the relationship between the indexes and their values at previous realizations, using both the actual values and the normalized (log) values. The results: p-values near to zero and R-squared values exceeding 90% were impressive but misleading.
Unfortunately, the use of OLS relies on the stochastic process being stationary (variances and covariances should not change over time). Housing price indexes are not stationary and have no discernible trends or cycles. We decided to focus our research on the largest 50 MSAD and used R to run a series of different tests to check for stationarity and potential unit roots. The presence of a unit root would signal the possibility of a “random walk”, meaning that random shocks are likely to have long-term effects on the series.
We initially applied the Kwiatkowski–Phillips–Schmidt–Shin (KPSS) test, used to verify the null hypothesis that the time series are stationary around a deterministic trend. It didn’t allow us to exclude the hypothesis of non-stationarity for the MSAD. Therefore, we tested the series for unit root through two different unit root tests:

  • The Phillips-Perron (PP) unit-root test, which builds on the Dickey-Fuller test, but introduces corrections to the limits the latter presents when dealing with serial correlation. PP tests can be run either including or excluding the trend component;
  • The Schmidt-Phillips (SP) test, another variant of the test that incorporates an estimate of the deterministic trend, and applies the unit-root test to the series adjusted for the deterministic trend terms.

First, we ran both tests for the normalized values of the indexes, and in both cases we found that none of the MSAD series are stationary. Thus, the null hypothesis of unit root could not be rejected, and it was necessary to apply the difference operator to the series.
Using the first difference, both the PP and PS tests indicate that 21 of 50 MSAD time series are stationary under the first difference. These metro areas are: Austin, Boston, Cambridge, Charlotte, Cincinnati, Cleveland, Columbus, Dallas, Fort Worth, Houston, Indianapolis, Jacksonville, Kansas City, Milwaukee, Montgomery, Nashville, Philadelphia, Pittsburgh, St. Louis, San Antonio, and Virginia Beach. Besides, the PP test showed significance of the trend component for 6 of these MSAD (Cincinnati, Cleveland, Columbus, Indianapolis, Kansas City, and St. Louis).

We ran a second set of tests on the remaining MSAD using the second difference. The results were, in some cases, contradictory. The Phillips-Perron test gave us significant value of the test-statistics for all the 29 MSAD (Anaheim, Atlanta, Baltimore, Chicago, Denver, Detroit, Fort Lauderdale, Las Vegas, Los Angeles, Miami, Minneapolis, Nassau County, Newark, New York, Oakland, Orlando, Phoenix, Portland, Providence, Riverside, Sacramento, San Diego, San Francisco, San Jose, Seattle, Tampa, Warren, Washington, West Palm Beach), and didn’t show significance of the trend component for neither of them. Whereas under the Schmidt-Phillips test we could not confirm the series were stationary under the second or third differences. These conflicting results can probably be attributed to the inclusion of the deterministic trend inherent in the SP approach.

The structure and characteristics of US housing price changes appear to be highly heterogeneous. While we couldn’t reject a null hypothesis of unit root for none of the 50 MSAD, we were able to confirm that for 21 of them it is possible to build a parsimonious model under the first difference. As for the remaining 29, we will need to analyze them further, to see if actual possibilities of forecasting exist. Specifically, if a difference when the process is stationary cannot be found, the results of any regression analysis with these values would be spurious.

Therefore, we have to question the practical usefulness of models that assume a single parsimonious representation for all HPI data. When attempting to develop a statistical model for US house pricing, researchers must take into account the heterogeneity.