Forecasting demand using Big Data

As we walk, cycle or drive around Oxford, make telephone calls, send texts or emails and do our shopping, many of us are unaware of exactly how much data is being generated by our activities. "Big data" is a catch-phrase for describing the overwhelming volume, velocity and variety of this stream of information. Big data has the potential to provide many opportunities for the public and private sectors, offering a means of fusing different sources of information and supporting decision-making in real-time.

Perhaps the most interesting aspect of big data is how it deepens our understanding of human behaviour seen through the collective actions of many individuals. We tend to consume services following the temporal cycles in our everyday lives. There are three evident cyclical patterns based around the hour of day, the day of the week and the season of the year. All of these patterns can be seen in electricity consumption, call centre activity, internet usage, financial transactions, traffic flow and the use of healthcare services.

Fortunately the repetition of these patterns offers potential for accurate demand forecasting. Services can be delivered with greater efficiency if staff and limited resources are scheduled in order to meet forecasted demand. The National Grid has been balancing supply and demand for years and knows the value of accurate forecasts. If they get it wrong, the lights go out and everybody notices. While power outages still happen in many countries, we take it for granted in the UK that we have reliable access to electricity at all times. Amazingly, we are relatively tolerant of imbalances in supply and demand in other sectors and this may explain why sophisticated demand forecasting is not widely utilised.

Take healthcare, for example. The NHS has a target of seeing 95% of patients arriving in A&E within four hours. Until recently there was little information about the performance of our local hospitals or indeed how they compare with the rest of the country. Now weekly A&E data about the percentage of patients seen in four hours is available. This week the John Radcliffe Hospital A&E scored 87.1%, slightly less than the national average of 91.5%. Here is a chance for Oxford City to become smarter.

There are many opportunities to use big data and quantitative models to forecast demand, develop early warning systems and improve staff scheduling. The graph below shows the average hourly A&E arrivals at the John Radcliffe for different days of the week. We immediately see the hour of the day effect with low demand during the night and two peaks at 12:00 and 18:00. Most striking is the near doubling of arrivals in the early hours of Saturday and Sunday, which can be attributed to the effect of weekend partying and pubs closing at 11:00 on Friday and Saturday night. While A&E staff are well aware of the additional burden caused by weekend festivities, the data analysis paints a clear picture of its impact on arrivals.
Big data can facilitate a better understanding of social behaviour and the effect of the environment. Arrivals increase on bank holidays. Temperature is another important factor with arrivals increasing in warm weather. But this is just the start. Forecasts of extreme weather events and information about social events could be used to construct an accurate model.

Comments

Post a Comment

Popular posts from this blog

Remote health diagnosis and monitoring in the time of COVID-19

Image
Coronavirus disease (COVID-19), caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), is rapidly spreading across the globe. The clinical spectrum of SARS-CoV-2 pneumonia requires early detection and monitoring, within a clinical environment for critical cases and remotely for mild cases, with a large spectrum of symptoms.  The fear of contamination in clinical environments has led to a dramatic reduction in on-site referrals for routine care. There has also been a perceived need to continuously monitor non-severe COVID-19 patients, either from their quarantine site at home, or dedicated quarantine locations (e.g. hotels). In particular, facilitating contact tracing with proximity and location tracing apps was adopted in many countries very rapidly. Thus, the pandemic has driven incentives to innovate and enhance or create new routes for providing healthcare services at distance. In particular, this has created a dramatic impetus to find innovative ways to remotel
Read more

Use of Machine Learning Techniques to Create a Credit Score Model for Airtime Loans

Image
  Airtime lending default rates are typically lower than those experienced by banks and microfinance institutions (MFIs) but are likely to grow as the service is offered more widely. In this paper, credit scoring techniques are reviewed, and that knowledge is built upon to create an appropriate machine learning model for airtime lending. Over three million loans belonging to more than 41 thousand customers with a repayment period of three months are analysed.  Logistic Regression, Decision Trees and Random Forest are evaluated for their ability to classify defaulters using several cross-validation approaches and the latter model performed best. When the default rate is below 2%, it is better to offer everyone a loan. For higher default rates, the model substantially enhances profitability. The model quadruples the tolerable level of default rate for breaking even from 8% to 32%.  Nonlinear classification models offer considerable potential for credit scoring, coping with higher levels
Read more

A racing heart for St Valentine's day

Happy St Valentine's Day A racing heart might be a sign of love or fear ... Using a mathematical model of the electrocardiogram produced by a human heart, we speed up the heart to produce some interesting music. Synthetic ECG generator: ECGSYN --> McSharry PE, Clifford GD, Tarassenko L, Smith L. (2003). A dynamical model for generating synthetic electrocardiogram signals . IEEE Transactions on Biomedical Engineering 50(3): 289-294; March 2003.
Read more