Impact of COV-2 on local crime statistics

A process for data gathering from ASP.NET Web app using form request engineering. Method is applied to gathering and analysis of police activity detail in the town of Bellingham, Washington pre- and post COV-2.

View of Mt Shuksan in Autumn from Picture Lake, Mt Baker Snoqualmie National Forest near Bellingham, Washington. Courtesy : U.S. Forest Service- Pacific Northwest Region

Introduction

Bellingham, Wa is one of the top destinations in United States for mountain biking and is a very fast growing town, thanks to its proximity to Seattle and Vancouver. I have lived here since 2017 and I love the town, its people, and the most of all the great access to outdoors.

For my day job, I work as an engineer who enjoys using data to solve problems. I have been curious for a while on the impact of COVID on crime rates in my town. During a random google search, I stumbled upon the treasure trove of data that can be leveraged to help answer this question. This is the City of Bellingham’s, police daily activity reports which go all the way back to 2009 upto the present (updated daily).

The City of Bellingham offers up its daily police activity report in a easy to access format. Very nice!

The daily activity report includes information on the timestamp of the reported incident, location (on a city block level), the offense category, case #, and unstructured information on details of the offense. I decided to collect these reports going all the back to 2015 to evaluate seasonal, and year-on-year trends in crime rates in Bellingham and, most importantly, if COV-2 has had an impact on it. The only challenge was that the online portal limits downloading of reports to 3 months increments. Easy with Beautiful Soup right? Modify the html link for different start and end months, send a request, and off we are to scraping. Unfortunately not….

The data portal provided by the City of Bellingham for access to the police activity reports is a ASP.NET form and needs some “extra” effort using requests to retrieve the html page consisting of the data which can inturn can be processed using Beautiful soup.

Part I: ASP.NET Webscraping Process

Since I am new to the dark arts of webscraping, applying to to ASP.NET was a challenge for me. I started with doing a “literature review” on existing methods available on pages of stack overflow and other medium posts. Unfortunately, I was not able to find a a clear process on how to approach such problems. This is the motivation for this post.

General process and methodology for ASP.NET data scraping and how I applied to this specific problem statement

After much trial and error (and many shots of espresso), I put together a simplified methodology that I have now applied to multiple ASP.NET scrapping projects with great success. I have presented the general methodology, the method applied specifically to scraping Bellingham’s police activity detail here (along with sample code). It is assumed that once you have the form output into Beautiful Soup, you are capable of data wrangling to get the output into required format for analysis (e.g. Pandas DataFrame)

How to apply steps 1,2,3 of the process using Chrome Developer Tools.

This code is a sample for how to engineer a form to programmatically scrape data from an ASP.NET Application.

Part II: Impact of COV2 shutdown on Bellingham, WA crime statistics

Once the data is in a dataframe, the analysis is pretty straight forward. I have included here the notebook I used to understand the impact of crime in Bellingham, WA between 2015–2021.

Jupyter Notebook to analyze output of scraped data.

So, what is the impact of COV-2 shutdown in crime in Bellingham…

Historical trends in crime in Bellingham, Wa and impact of COV-2 shutdown.

Based on the police activity detail data, the impact of COV2 related shutdown has generally reduced total crime in Bellingham, WA. However, the trends are different depending on the category of crime. More specifically,

1. There is a slight reduction in property, and statutory crimes.
2. Significant reduction in violence and drug-narcotics related crimes.
3. No statistically significant difference in death investigations (including homicides).

Interestingly, the last point is consistent with observations of other studies on the impact of COV2 shutdown on crime . According to one nation wide study,

the overall crime rates, as measured by call for service to law enforcement, have decreased markedly. However, the crime drop seem to hiding a very disturbing trend where homicides remain unchanged and possible increasing [1].

Conclusions

In this article, we looked at how to engineer a form to scrape data from a ASP.NET application and applied to retrieving data from City of Bellingham’s police activity detail.

1. For simple ASP.NET web apps, the python request and beautiful soup library are all the tools you need to engineer a data pipeline. For advanced applications, we can use selenium library with chrome driver (to be covered in a different post).
2. Based on police activity detail data, the overall crime rate in Bellingham, WA has significantly reduced as a result of COV2 shutdown. However, no statistically significant conclusions can be drown about serious crimes such as homicide. This is consistent with nation wide crime studies.

These findings here are a result of simple statistical analysis.

Can you help improve the analysis?

Can you help improve the analysis? Data can be found Github link provided below.

References

Has COVID-19 Changed Crime? Crime Rates in the United States during the Pandemic

varunr89/cov2_crime_housing