While machine-learning techniques can improve business processes, predict future outcomes, and save money, they also increase modeling risk because of their complex and opaque features. In this article, Milliman’s Jonathan Glowacki and Martin Reichhoff discuss how model validation techniques can mitigate the potential pitfalls of machine-learning algorithms.
Here is an excerpt:
An independent model validation carried out by knowledgeable professionals can mitigate the risks associated with new modeling techniques. In spite of the novelty of machine-learning techniques, there are several methods to safeguard against overfitting and other modeling flaws. The most important requirement for model validation is for the team performing the model validation to understand the algorithm. If the validator does not understand the theory and assumptions behind the model, then they are likely to not perform an effective model validation on the process. After demonstrating an understanding on the model theory, the following procedures are helpful in performing the validation.
Outcomes analysis refers to comparing modeled results to actual data. For advanced modeling techniques, outcomes analysis becomes a very simple yet useful approach to understanding model interactions and pitfalls. One way to understand model results is to simply plot the range of the independent variable against both the actual and predicted outcome along with the number of observations. This allows the user to visualize the univariate relationship within the model and understand if the model is overfitting to sparse data. To evaluate possible interactions, cross plots can also be created looking at results in two dimensions as opposed to a single dimension. Dimensionality beyond two dimensions becomes difficult to evaluate, but looking at simple interactions does provide an initial useful understanding of how the model behaves with independent variables….
…Cross-validation is a common strategy to help ensure that a model isn’t overfitting the sample data it’s being developed with. Cross-validation has been used to help ensure the integrity of other statistical methods in the past, and with the rising popularity of machine-learning techniques, it has become even more important. In cross-validation, a model is fitted using only a portion of the sample data. The model is then applied to the other portion of the data to test performance. Ideally, a model will perform equally well on both portions of the data. If it doesn’t, it’s likely that the model has been over fit.
Liquidity risk is one of those risks we often don’t pause to think that much about, but it’s one that can wreak havoc on a business if not kept constantly in check. It is also a risk that has become heightened in recent times, because of a combination of regulatory and macroeconomic developments. Companies can often grow complacent about liquidity risk, especially if they have tended to generate cash on a consistent basis through ongoing operating performance. However, certain activities, such as mergers and acquisitions (M&A), a new product launch, or perhaps regulatory development, can give rise to new exposures. It’s worth reminding ourselves of some of the key drivers of exposure to liquidity risk, and what we can do to manage and mitigate this risk.
In Europe, the ability to recognize negative best estimate liabilities on the solvency balance sheet, effectively capitalizing estimated future profits on books of in-force business, and considering these profits to be immediately available to absorb losses in the business, requires companies to be extra vigilant. In reality, such assets may be far from liquid, unless they can be repackaged through value-in-force (VIF) monetization, or used to secure reinsurance financing of some sort. The same may be said of the likes of deferred tax assets, except that these assets may be even less liquid, unless they can be sold on to other entities within a group structure.
Other aspects of the liability side of the balance sheet can also pose liquidity challenges. Take, for example, a company with a range of unit-linked funds operating on a t+1 basis (i.e., settlement occurs one day after the transaction date), with a further range of funds operating on a t+2 basis. Policyholder fund switches out of the t+2 funds and into the t+1 funds can leave companies needing to provide liquidity for transaction settlements upon purchase of the t+1 assets, before payment is received from the sale of the t+2 assets. Depending on the volume of transactions, which could be significant, firms may struggle to provide such financing on an ongoing basis. More severe examples of firms struggling to cope with fund switch activity have included suspension of redemptions from property funds, albeit more because of the underlying lack of liquidity of the assets than the nature of the pricing basis, though ultimately leading to similar problems. Funds that permit a mix of both individual and corporate investors may be particularly susceptible, as the latter potentially have the ability to move vast sums of money very quickly, and before redemptions are suspended, precipitating the lack of liquidity for individuals.
Milliman today released the 2016 Retail Cost of Risk study, which measures the cost of property and casualty risks faced by retailers. In 2016, the average cost of risk for U.S. retailers is projected to amount to $5.93 per $1,000 of sales. Around 60% of the costs is attributable to the injuries of employees and customers; workers’ compensation costs represent around 40% of the overall risk and the cost of customer injuries represents around 20% of the total. Other risks, including property damage, cyber, and miscellaneous liabilities, account for the remaining 40%.
Back-to-school and back-to-college shopping represents the second-largest shopping season of the year, with projected sales of $75.8 billion in 2016, according to the National Retail Federation. The many families still wrapping up their back-to-school shopping lists may not be aware that retailers devote a small but significant share of every dollar you spend with them to pay for their risks. This year, roughly one-in-200 households will be involved in a general liability incident at a retailer. Most of the incidents are small property damage claims such as damage to vehicles in parking lots. In addition, the more than 15.7 million people employed in the retail sector are expected to experience over 55,000 workplace injuries each month. As with customer injuries, the vast majority of such incidents will be minor in nature.
Milliman has launched the Retail Cost of Risk study in an effort to illuminate the risks that retailers face, and to track how they change over time, especially as new liabilities—cyber risk, for example—become a growing and more expensive concern for the retail industry.
Retailers work hard to reduce the potential for employee and customer injuries. The frequency of workers’ compensation and general liability claims has consistently declined since 2011, a testament to the success of safety and risk management programs. Despite the decreasing claim frequency, the cost of risk has remained fairly level over time (as a percentage of sales) because the average cost of such injuries has risen each year, and because retailers face new risks. For example, retailers saw a 40% increase in the cost of cyber insurance in 2015, not to mention an increase in deductibles, which puts more of the risk on retailers.
To download the complete Retail Cost of Risk study, click here.
Pikachu and his friends have caused quite a frenzy recently. While people are enthralled with Nintendo’s Pokémon Go, the GPS-based augmented reality (AR) game presents several risks to its developer and its gamers. In his article “Pokémon Go and augmented reality: Not all fun and games,” Milliman consultant Michael Henk discusses some of these AR technology-related risks.
Concerning personal injury risks:
Firstly, AR products like “Go” provides yet another “distraction.” We’re all aware of the dangers of being “distracted.” Texting while driving is illegal in a number of cities and states throughout the country. However, drivers aren’t the only ones being distracted. Distracted walking is a growing problem, one that has arisen naturally with the increasing dependence on mobile electronic devices and one that “Go” is already contributing to. There are anecdotes all over social media about players so engrossed in catching virtual monsters that they’re running into walls and walking in traffic. …
…“Go” may lead to an increase in distraction-caused injuries and pedestrian-vehicle injuries, which is currently the fifth-leading cause of death for children ages 5 to 19. It’s not inconceivable to imagine an incident in which both the driver and the pedestrian are distracted, maybe by the same “rare” Pokémon.
What about cyber risks?
Aside from “IRL” (in real life) dangers, there’s a data security concern with some early installs. Some iOS installs of the software require the user to provide the app with full access to their google accounts, which allows access to their Gmail (theoretically being able to send e-mail from your account), files stored on Google Drive and Google Photos, among other content. The developer has responded and said this was done erroneously, and that permissions will be corrected soon, but it’s important to make sure that users know exactly what programs on their devices have access to. There are other concerns about downloading the program from non-official app stores as well, but that stands for all programs and is definitely not a “Go”-specific concern.
…There’s a significant risk for trespass with AR games that utilize real-world locations. It remains to be seen whether an AR developer placing cyber-content on your property constitutes trespassing or if AR users are “engaged on a cyber plane on which you have no exclusive property claim.” There’s another legal concern with “attractive nuisance,” which states that property owners are responsible for eliminating dangerous conditions on their property which may attract children. “An individual who fails to rectify an attractive nuisance on their property is civilly-liable to injury a child sustains on it, even if the child was trespassing.” Sounds like something that may happen in the pursuit of a rare Pokémon.
This latest edition of Milliman Impact entitled “Bolstering insurers’ cyber defences” explores the efforts of U.S. insurance regulators to address cyber security risks.
Here’s an excerpt:
Unsurprisingly, insurer cyber security has become an important issue for US regulators in recent years.
In the spring of 2015, the New York insurance supervisor wrote to more than 160 insurers encouraging them to view cyber security as an integral aspect of their overall risk management strategy. It also announced enhancements to the IT examination framework to include more detailed questions on an insurer’s cyber security policies, protections, and procedures.
More significantly, the NAIC has engaged in a burst of activity, having taken the significant step of establishing a Cyber Task Force in November 2014.
Creating the task force demonstrates US insurance supervisors’ commitment to addressing cyber security in the insurance sector, according to Christine Fleming, claims management consultant at Milliman in Boston….
The task force’s comprehensive work plan and timetable speaks volumes to the significance and urgency that US insurance supervisors and commissioners now place on cyber security, explains Fleming.
The task force is concerned with both the protection of consumer data held by insurers and improved monitoring of insurers cyber underwriting activities and exposures. During 2015, the NAIC embarked on four major work streams:
• Establishing guiding principles on cyber regulation
• Creating a Consumer Bill of Rights
• Modernising examination protocols to include cyber security
• Including a cyber security statement in insurers annual statement
Cyber risk is destined to become a much bigger part of insurers’ business yet it also comes with challenges. Insurers are grappling with a lack of historical data and the threat of aggregation and systemic risks. At the same time, regulators and ratings agencies are becoming more aware of the potential risks that cyber presents to insurers’ balance sheets and the industry’s reputation. This Milliman Impact article provides more perspective.