Category Archives: data analytics



Insurers and Wealth Managers have long aspired to segment customers based on their life stage. Big Data can finally make this happen

by Ai Meun Lim, Chief Product Officer, Percipient, and Dr Shidan Murphy,  Senior Data Scientist, Angoss Software Corporation

Shakespeare describes the “The Seven Ages of Man” as the Infant, Schoolboy, Lover, Soldier, Justice, Pantaloon (roughly translated as foolish old man) and Old Age. While today our view of one’s life stage is perhaps less cynical (and poetic!), the desire for such a clear classification remains a strong business goal across a range of industries.

Most businesses implicitly recognize that an individual’s life stage contributes towards a multifaceted perspective into the needs of a customer. Further, in some tightly regulated financial businesses, such as wealth management and insurance, determining a customer’s life stage is required to determine risk tolerance and financial objectives. Understanding a customer’s life stage is essential for any customer-centric business strategy.

However, despite the importance, most financial institutions have failed to adequately monetize the concept of a life stage. Major limitations to such monetization are:
– Outdated perceptions of a customer life stage.
– Unavailable or outdated customer data
– Inability to track customer life changes
– A lack of predictive analytics

Why a Dynamic Segmentation of Life Stages?

Traditional methods have based life stage assessments on easily quantifiable factors such as age and income. Yet an age or income-only lens has proven inadequate for detecting financially-important events such as getting married, starting a family, launching a business venture, or taking care of elderly parents.

While the automatic and fluid tracking of a customer’s life stage is sought after in financial institutions today, most still rely on ad hoc updates from customers themselves, and at best, annual customer surveys. Such passive tracking methods point to difficulties financial institution have meeting their “duty of care” obligations, let alone running profitable and targeted life stage campaigns.

The development of a dynamic segmentation of life stages is therefore a generational-leap in customer understanding. By digesting real time data from traditional and non-traditional data sources and using advanced algorithms to analyse this data, it is now possible to create a granular customer segmentation that is capable of evolving with the customer’s lifecycle.

Life Stage Analytics

Percipient and Angoss are collaborating to offer dynamic life stage segmentation that applies the tools capable of unifying such data, and technological advances in big data analytics.

The foundation of this approach is data – and lots of it. For the segmentation to be meaningful, the data must include both conventional and unconventional data sources.  Conventional sources include demographic data, spending patterns, and both assets and liabilities. Although these data are readily available in financial institutions, they are often underutilized.

Advancements in data technology means there is also scope to incorporate unconventional data sources such as social media (think LinkedIn), wearable devices (think Fitbit), digital footprints and third-party data aggregators.  Some financial institutions may already be collecting this data, but have not been put it to use for this purpose.

Customer segmentation is then created using cutting-edge analytics. The analytical approach combines business rules and next-generation tools and techniques to create granular-level customer categorizations. The life stage categorizations are dynamic and regularly updated to reflect changes in the data feeds.

Such dynamic categorizations create the opportunities for, among other requirements, next product recommendations, customer retention and calculating the actual and expected profitability of the customer base.

A Revolution in Needs Based Selling

The financial industry’s mantra is that product sales be underpinned by their customer’s life stage. Today, data and next-generation tools are in place to create more accurate, dynamic and granular view into the needs of their customers. An accurate, analytics-driven understanding of life stages are set to revolutionize needs-based selling.


2016 Revelations


Big data can be funny too…no…seriously! 


As the year draws to a close, and we treat ourselves to some well-earned merry making, here is a look back at some 2016 big data events that prove big data isn’t all dull and serious.

A World of How, What and Why


Google’s annual list of most googled terms are always revealing and this year’s is no different. Australia’s Top Ten “How to” searches in 2016 suggest that Australians are still concerned with the challenges of daily life: “How to tie a tie?” and “how to get rid of pimples?”

UK-based Googlers, on the other hand, were concerned with a somewhat more esoteric question: “How to make slime?”. Their Top Ten also included the more sadly philosophical: “How to accept myself for who I am?”.

Meanwhile, the “Why” question uppermost with Swedish Googlers in 2016 was “Why are eggs brown or white?”. Clearly, the world is still full of innocents.

 (Extra) Ordinary Gifts


This year’s Singles Day has been deemed Alibaba’s most successful yet. Sales this year reached USD 17.79 billion, compared to USD 14.3 billion last year. Unsurprisingly, top sellers were phones and appliances.

But everyday household items are also big hits on Singles Day. Hera BB cream from Korea and Laurier sanitary napkins from Japan are traditional favourites, as is milk.

Last year, one German supplier alone accounted for 2.35 million litres (USD14.3 million) of Singles Day sales of liquid milk. On the same auspicious day this year, an Australian manufacturer was able to shift 350,000 goat soap items worth over USD 1 million.

Deep Learners?


Alongside top trending searches, Google’s 2016 Breakout Searches (ie yoy searches that rose by 5,000% or more) were equally eye catching.

While “Severe Weather” featured in Germany’s list, “Nobel Prize” in Sweden, and “Melania Trump” in Slovenia, China’s list contained only two: “Deep Learning” and “Machine Learning”.

This presumably has something to do with ordinary people wanting to understand what organisations – from the government to corporates – are doing with their data. For example, e-retailer has determined that Chinese consumers prefer to shop for underwear in the late evening. While it is still unclear how this will drive future marketing, the retailer says such “subtleties” provide valuable insights into the Chinese psyche.

 Better Restroom Service


2016 also provided some insight into Japanese restroom habits and preferences. A report by the Japan Times revealed that highway operator, Central Nippon Expressway Co. (NEXCO Central) had installed 3,000 sensors, including motion detectors for toilet bowls, in 51 restroom locations along the Shin-Tomei Expressway.

Analysis of the data collected suggest that cubicle use is gaining on urinals, and an average cubicle visit is four minutes and four seconds, up 35 seconds in seven years, which NEXCO puts down to the increased use of mobile phones in restrooms.

NEXCO expect this data to help it improve its service. For male commuters faced with long cubicle waiting times, this will be a relief.

Its Been Weird


Spotify dug deep into its data stores to come up with one of the year’s most interesting end-of-year campaigns. Displayed on billboards at select locations around the world, Spotify highlighted the behaviour of some of the site’s users, and their reaction to it.

Under a general tagline of “Thanks, 2016. Its been weird”, examples included:

  • Dear 3,749 people who streamed “Its The End Of The World As We Know It” the day of the Brexit vote. Hang on in there.
  • To the 1,235 guys who loved the Girls’ Night playlist this year. We Love You.
  • Dear person who played “Sorry” 42 times on Valentine’s Day. What did you do?

Developed by Spotify’s in-house team, Chief Marketing Officer, Seth Farbman said their data was inspiring and gave an insight into people’s emotions.

We agree – where can you detect more emotion than in a person’s playlist?.

To personalise or not: WhatsApp re-ignites the data sharing debate

WhatsApp announced last month that it will allow its parent company Facebook to sell its user data to advertisers. The news was met with widespread consternation.

The timing too couldn’t be more unfortunate, coming just a fortnight after a landmark US Court of Appeal ruling in favour of Microsoft’s bid not to hand over its customer emails to the US federal government.

Also playing on many minds is the EU’s General Data Protection Regulation (GDPR), adopted in April this year after four years of debate, and due to come into force by mid-2018. Corporations located in EU member states are already scrambling to ensure compliance with requirements that are variously described as “onerous”, “radical” or “doesn’t go far enough”.

As both the private and public sectors struggle to draw the line between data privacy and data utility, it is perhaps time for individuals to ask themselves the same questions. What do I stand to gain from government and corporate use of my data? And when does this cross the line?

This is not just a philosophical exercise. In fact, data privacy advocates strongly espouse the concept that an individual’s personal data belongs to the individual. Governments and corporations are deemed “temporary custodians” of the data they collect, and can choose to offer personalised services, but based only on the information that individuals are willing to share with them.

Billed as a first of its kind, the high profile MyData 2016 conference taking place in Helsinki this week promotes this view of “human-centric data management”. Organisers of the event take pains to stress that their intention is not to stifle innovation, but rather to lay the ground rules for the ethical use of data. However, implicit in the conference themes is the notion that the scales may now have tilted too far in favour of the organisation versus the individual.

Aside from the issues of data security (ie keeping data safe from hackers and unauthorised persons), research suggests that individuals greatly fear the improper use of big data to drive key decisions made about them. Media, internet, telecommunication and insurance companies are said to face the greatest “data trust deficit” and need to make the most effort to ensure that their brand is associated with data transparency and accountability.

On the other hand, individuals should not under-estimate the extent to which big data mining has become an expectation. Research by the Aberdeen Group suggest that 74 percent of online consumers actually get frustrated with website offers and promotions that have nothing to do with their interests. The research also found that more than half of all consumers are now more inclined to use a retailer if it offers a good personalised experience.

The key appears to be control. According to, more than 60% of online users, while valuing personalisation, sought to understand how websites select such content. A similar number wanted the ability to influence the final results by proactively providing or editing personal information about themselves.

However, it is not marketing websites but IoTs and wearables that will be the biggest test of users’ embrace or disaffection with big data. Today’s low cost wearables are effectively subsidised by the potential monetisation of the data that these devices are able to generate. This data is vast and highly personal, and while the IoT trend is not new, experts expect wearable technology to escalate dramatically over the next few years.

It is therefore increasingly incumbent on the industry to put in place measures that protect the data from abuse. Manulife’s MOVE and AIA’s Vitality programmes are classic examples of how it is possible to align individual and corporate objectives to the benefit of both. Individual policy holders are provided with fitness trackers that record workout data, and rewarded for healthy behaviour through discounts or points. This is done by ensuring workout data is explicitly collected and consent explicitly provided, with a firm undertaking that the data will not be shared or used for other purposes.

It is said that with great power comes great responsibility. Big data has the power to transform organisations and disrupt industry practices. But in delivering the personalisation that individuals now demand, organisations cannot lose sight of their responsibility to maintain the individual’s innate desire for self-determination, even in the digital world.

Four Myths About Real Time Analytics


Five years ago, businesses beyond the e-commerce world were only dipping their toes into real time analytics. Today, brick and mortar businesses have embraced real time analytics in a range of applications, including error detection, price adjustment, inventory tracking, customer experience management, and more.

So what has changed? Certainly, advances in both hardware and software have helped. But at the heart of this reassessment is the data industry’s ability to dispel a number of myths surrounding real time data processing and analytics.

Here are four of the most deeply entrenched:

  1. Real time intelligence is only a nice-to-have

 A common misunderstanding about real time analytics is that it is the same as batch analytics, just conducted over a shorter time frame. Therefore, as long as important business reports are meeting required deadlines, intelligence in real time is regarded as less than critical.

In fact, in contrast to the analytics conducted on batch (ie static) data, real time analytics is not designed to diagnose key business trends. Rather, real time analytics is performed continually on streamed data in order to detect unusual events as they happen, and to trigger an immediate or quick response.

As such, real time analytics is best suited for processes which stream large volumes of data. Where the analysed data is signalling something amiss, fast action – usually dictated by pre-set business rules – can prevent it from escalating into a full blown problem.

For this reason, real time analytics’ greatest value is its ability to avert losses, for example, by detecting financial fraud or manufacturing defects. There is also the potential to improve routine operational efficiencies, such as through retail inventory management or telecoms network optimisation. Real time analytics is therefore deemed to deliver “operational intelligence”, and is complementary to, rather than a substitute for, “business intelligence”.

  1. The prices of my products/services do not change very often

Businesses are increasingly using real time analytics for another innovation – real time pricing. E-commerce sites have long used real time factors and complex algorithms to drive active price adjustments. What is perhaps less well known is the magnitude and velocity of these adjustments. Amazon, for example, changes the prices of about 40 million products many times during a single day, according to Internet Retailer Magazine.

The confidence that e-commerce sites have in their pricing models is now reflected in many traditional businesses. A report in the Wall Street Journal noted that, for example, adult passes to Indiana Zoo can range between USD 8 and 30, by responding in real time to weather changes, school group bookings, surprise closure of an attraction, and a variety of other factors.

Many other facilities now adopt similar real time pricing mechanisms, including highway tolls, parking lots, taxi services, golf courses, ski resorts, theme parks, and entertainment events. In the case of highway tolls, prices can fluctuate by as much as 500% in a single week. This real time approach to pricing is said to be capable of lifting revenues by 10 to 20%.

However, real time pricing offer benefits beyond that of higher revenues. Bottom line consistency and customer experience are often as important, with dynamic price adjustments used to control demand peaks and troughs, and associated resourcing costs. The question traditional businesses need to ask themselves is therefore not ‘How often should we review our prices?’ but rather ‘How can we approach pricing differently?’

  1. Real time information is unsubstantiated and therefore less valuable 

The term “real time” gives the impression of fleeting observations that are not verifiable. However, in most cases, real time analytics is not simply based on what has just occurred. Rather, such analyses involves correlations between real time and historical datasets, thereby placing a real time event in its proper context.

Take for example how a GPS now re-routes vehicles in order to avoid traffic congestion. The analyses required to do this is based not just on real time traffic information, but on the correlation with previous congestion events, in order to determine the best alternative routes that vehicles should take.

It is the ability to combine streamed real time data with historical data and data from other sources that gives real time analytics its greatest potency.

  1. Real time analytics is too expensive to implement

 Real time analytics is not without its complexities, but does not necessarily require a large business investment. This is because the evolution of open-source data technology has resulted in significantly more cost-efficient real time software. Spark, for example, is an open source project that is set to underpin much of the industry’s real time analytics capabilities.

In addition, rather counter-intuitively, the data architecture stack required to support the integration of real time feeds with existing batch based data processes can actually be a cheaper alternative to current storage technologies.

This is because the architecture needed to run real time analytics eliminates existing storage inefficiencies. Adoption of these stacks, which is entirely reliant on open source platforms, enables analytics to be performed incrementally and in memory, while also avoiding the need for large hardware investments.

Today, there are few excuses left for businesses not to adopt real time analytical capabilities. While the insights generated by analysing historical data can help businesses arrive at strategic decisions, real time analytics produce a different kind of insight. They are the triggers that enable business to tweak processes and refine experiences.

Healthcare and Health Insurance: Big Data’s Sweet Spot


Nowhere is the utility of big data greater than in healthcare, and by inference, health insurance. Venture capitalists appear to have caught on to this, and in the first quarter of 2016, put USD 1.4 billion into the hands of healthcare IT companies. This was almost a third more than in the previous quarter, with much of it directed towards wearables, data analytics and telemedicine.

#1: The Cost Challenge

Three key trends are driving the big data revolution in healthcare. The most important of these is the desperate need, in a world that is rapidly aging, to stem ever-rising healthcare costs. This is a need shared by all healthcare stakeholders, from consumers and providers, to insurers and government agencies.

While cost saving is not a new item on the global healthcare agenda, the recent spotlight on “value-for-money” health services is in part the result of the U.S.’s Affordable Care Act. Faced with ACA requirements, “merger mania” has broken out among healthcare insurers, according to the PwC Health Research Institute.

It would be fair to say that these proposed insurance mega-mergers have been met with some scepticism. Opponents fear that the much-touted economies of scale may not materialise. However, proponents and opponents alike agree that insurance company mergers do offer a key opportunity: to optimise their IT infrastructure, and leverage on innovations in mobile health applications and analytics.

#2: Monitoring on the move

Fitness Coach USA Running

This brings us to the second unmissable trend – the rise of activity trackers and medical devices. Increasingly, health monitoring is shifting from the hospital and clinic to the home and on-the-go. Given that the management of chronic diseases is already the largest component of U.S. healthcare spending, Goldman Sachs suggests that remote health monitoring is “at the bulls-eye of the healthcare cost challenge”.

But the power of health monitoring devices is not just the ability to save money. Health and fitness monitoring devices are enjoying a huge surge in popularity. According to IDC, 78.1 million wearables were shipped in 2015, up from 28.8 million the previous year. PwC’s recent survey suggests that one in three American consumers now have at least one medical, health or fitness app on their mobile devices.

What this means is that the healthcare industry is no longer just reliant on clinical data. Health-related digital data is now (potentially) on tap from medical and fitness devices. The question is whether the data is being put to good use. To date, many healthcare and device providers have been reticent to do so, amid concerns that personal data sharing would be unpalatable to patients and users. The PwC survey appears to lay this to rest, with 83% of respondents indicating their willingness to share their data to help their own diagnosis and treatment, and 73% to help others.

#3: Prevention not cure


This level of acceptance may stem from the already signifiant strides in healthcare analytics, particularly in the area of patient risk forecasting. For example, by analysing patients’ everyday behaviours, socio-economic factors, and clinical updates, hospitals are now predicting and where possible, preventing, their post-operative health deterioration. It is perhaps not surprising that, in a recent survey, advanced analytics was found to be more prevalent among mid-sized U.S. hospitals, ie those between 100 – 500 beds, where there is a strong incentive to minimise patient re-admissions.

Furthermore, in the last few years, healthcare analytics have expanded to include healthy individuals with specific health vulnerabilities. Individuals at risk are offered wearables that track their vital statistics and surroundings, and analytics help primary care doctors to determine if a change in preventive medication or lifestyle is required.

Then there are the individuals who are tracked and rewarded simply for staying healthy. Employers, partnering with insurance players like United Health, Kaiser Foundation Group, Humana Group and Aetna, now account for the fastest sales of wearables. By agreeing to have their activity levels monitored and analysed, employees are able to lower their health insurance premiums as part of so-called “corporate wellness programs”.

A data-hungry future


Given the pace of adoption of wearables and healthcare analytics, there is an appetite for data that may well grow to point of becoming insatiable. Already, insurance companies are said to be busy approaching app developers in an attempt to directly access their fitness data.

By applying sophisticated analytics to large and varied datasets, insurers are hoping to refine their insurance plans. On the horizon looms a cut-throat health insurance marketplace, with new players such as hospitals and primary healthcare providers joining the fray. To stay competitive, health plans could evolve from annual and generalised premiums to daily-adjusted, personalised rates, based on an individual’s real time fitness data. This data will be generated by devices that insurers will offer at discounted rates or for free, further elevating data volumes and the performance of healthcare analytics.

Can you think of a more compelling virtuous cycle?