I recently reread “Extreme Risk Management. Evolutionary Approaches to Evaluating and Measuring Risk” by Christina Ray.
It’s a great read and one I highly recommend.
Christina believes that the intelligence community and financial communities face essentially the same issues when it comes to anticipating and managing the risk of extreme events (think collapsing currencies, sovereign defaults, bank runs etc).
She argues that the financial industry should be using the technologies adopted in National Security. By using real time market intelligence (“MARKINT”) as part of an all source intelligence strategy, risk managers can make sense of volatile markets in the present and better anticipate extreme events. Rapidly changing, market dynamics combined with an increasing number of complex finanical instruments challenges traditional historically based prediction models to perform accurately. We argue that new approaches are needed to generate actionable intel ahead of the financial equivalent of a terrorist threat.
From Saffron’s direct experience in National Security, I agree with Christina that the Financial Industry is not yet fully aware of just how far sense making, MARKINT style sensemaking technology has come.
Persistent, extreme risk is our new business landscape, and may well be for the foreseeable future.
It is the ability to unify numerous structured and unstructured data sources in real time, combined with a sense making tool and analytic reasoning methods that learn about, associate and connect new events with your interests – also in real time – that allows decision makers to anticipate and manage the risks around future black swan events.
People are looking for the edge, where is the information that I can leverage to get ahead of the rest of the market? Unstructured data seems to be the untapped source, whether it is news, social or both.
The Financial industry clearly calls out for a new approach to predictive analysis or what Saffron refers to as anticipation or “anticipating the future”. Prediction in this industry has been characterized by the use of a massive amount of historical market data reduced to models about the past to create forecasts about the future. But when the present is characterized by the increasing complexity of financial instruments and the increasing volume, variability and velocity of available data, then models based on the past become inadequate, uninformed and misguided.
Some news organizations, like Reuters and their work with OpenCalais, have been trying to help by bringing unstructured text into this analysis. OpenCalais automatically “reads” Reuters content and marks up the people, places and things (entities) that are being written about in the news article.
Dow Jones takes a similar approach to entity markup and makes these details available to algorithmic traders for analysis. In the past, accurately representing the people, places and things in data has been a big challenge.
But the question remains, even after the markups are done, how do we capture all of this knowledge and better exploit it to inform decision-making and help us anticipate the future?
To this end, change is underway on a variety of fronts. Take the recent sentiment analysis of Twitter and the news to help anticipate socio-political change. By analyzing public sentiment, as expressed in text, you can identify changes in public opinion or attitude. Arab Spring is a good example:
A new study by Computer scientist Kalev Leetaru, at the University of Illinois at Urbana-Champaign, claims you could have foreseen the Arab Spring if only you’d been paying enough attention to the news. Leetaru shows how data mining of news reportage can reveal the possibility of future crises well before they happen. Leetaru mined news reporting to identify the sentiment, as reported, on Tunisia, Egypt and Libya. “In all cases”, he says, “there was a clear, steady trend towards a negative tone for about a decade before the revolts.”
Monday morning “quarterbacking” is always easier. Leetaru knew that long-established dictatorial political leaders had been deposed. In other words, he knew what to look for in this case. Despite that, his argument is that sentiment analysis can be an indicator of an impending event, crisis or opportunity.
The ability to analyze unstructured data sources is an important component of analysis, such as anticipating socio-political unrest or mitigating brand reputation risk or determining the best approach to replacing damaged non-inventoried components. Incorporating unstructured data sources in business decision-making is proving to be a viable component of business intelligence. It provides context, facts and additional insight about a topic or transaction that cannot be expressed by predefined structured data.
The automatic combination of text or unstructured data with structured data creates a new, unified knowledge source. This marriage of structured data with unstructured sources not only changes the way analysis is done but creates a more informed, more complete and more accurate knowledge base to exploit for decision making and anticipation.
Without this, text analysis systems are just saying that the overall sentiment toward Egypt is bad (and you know this by reading and looking at the overall trend).
But a deeper understanding of the current (and past) relationship of people, places, things and events makes anticipating potential outcomes possible and thereby creates actionable, informed intelligence for risk management, reputation management, and a variety of business decisions.
In our personal finances, we want to know “should I invest in this stock or not? Should I divest of this stock or not? Should I stay in equities or move to cash?”
Elevating business intelligence to answer such questions cannot be accomplished by just analyzing unstructured text sources.
At Saffron we’ve created a new school of business intelligence that combines text sources with more traditional structured, transactional data sources. We do this through a specialized data index based on associations. Saffron stores the semantic structure of your data, along with frequency counts so that relevance metrics can be computed. Now you have an agile knowledge source to exploit, in real time, for decision support and anticipation applications. You can now answer questions such as “which of our customers will churn given a competitors new product announcement? Who are these customers? Why will they churn? What has happened to provoke churn?”, without predefined models or rules, and instead using all the information available about a customer, your interactions with them, all of their transactions with you and ultimately their interests and concerns as expressed in their communications with you.
Understanding how and why markets and customers move as supported by a variety of real time and historical data about indicators, transactions, socio-political events, and public sentiment is the Holy Grail of Analytics. Bringing all this together into a naturally predictive system is the Holy Grail of the Financial Industry.
On second thought, isn’t this the Holy Grail for all industries?
Imagine a Decision Support tool where you have, at-the-ready, your entire organization’s Experience about all prior decisions your company has ever made – and their outcomes – good, bad or ugly.
With Saffron … you do.
Saffron helps you squeeze every ounce of prior Experience possible from your data – across your enterprise knowledge base, or residing in your business processes – and considers it on your behalf so you can make quicker, more informed decisions today.
Saffron Natural Intelligence Platform
The Saffron Natural Intelligence Platform operates in a dimension beyond the limiting rules and models of traditional business intelligence or data analytics tools. What’s different is Saffron works by memory based reasoning. That means it identifies all the attributes for a specific decision case, and quickly identifies other similar cases, so you can quickly know as much as possible – and use your prior experience – on the matter about which you’re making a decision.
Saffron’s history with Decision Support applications is extensive. In National Security work we’re involved with Intelligence, Surveillance and Reconnaissance operations. In the Manufacturing sector, we’re helping generate replacement optimization recommendations for non-repairable parts; and in Banking & Finance, we’re supporting deep analysis of suspended mortgage loans to identify best actions to shorten approval cycles, to name just a few.
How it Works
It’s very straightforward.
We start by understanding what you want to make decisions about, and the data sources you want included in the analysis. By identifying the known possible outcomes, such as “go/no go,” “approve/deny,” or for more complex decisions, multiple choices. By computing all the likely outcomes, showing similarities among prior cases, and identifying key factors, Saffron presents you with just the data you need to make more informed decisions. And as you make them, the system learns and remembers, enriching its capabilities for future use.
Here’s an example of how Saffron helped a customer company in the mortgage loans industry to apply Experience Management by Decision Support. The customer tried for two years, but failed to create a new suspended loans decision support tool using traditional rules-based and statistical methods. They then approached Saffron to discuss applying our associative memory technology to the problem. Their story follows.
Industry Use Case
Mortgage Loans Processing / Suspended Loans
In mortgage loans processing, suspended customer-loan applications can grow into lost revenue, costing banks a lot of money. Some customer-loan applications are suspended due to incomplete or inconsistent information; others because of problems with the property itself. But still others are suspended for reasons amounting to a “mismatch” with the approval rules, as pre-defined by the company’s loan processing software.
Mortgage bankers want suspended loans to re-enter the approval process as quickly as possible, both to insure good customer service and to achieve optimum revenue for the mortgage loan company. To do this, suspended loans must be carefully examined to determine how best to correct these inconsistencies.
“We tried for two years to solve this problem–and failed. Until Saffron.”
— Financial Services Company
Decision Support Objective
Use the collective Experience of mortgage bankers, to reduce the volume of loans suspended and not closed in the mortgage loan approval process, empowering individual bankers to make faster revenue-impacting decisions.
A successful, well-respected global mortgage loan company processes a high volume of mortgage loans daily. Their goal is to ensure every applicant is correctly evaluated in their pursuit of a mortgage loan. All loans are subject to strict rules in a well-defined loan approval processing system.
Many mortgage loan applications fall out of the approval process for failure to meet the “rules’ of the existing loan approval management system. When this happens a customer’s loan application is suspended. Thousands of loans can be in a suspended mode at any one time for a variety of reasons, e.g., applicants may forget to report all their income, or leave important information off their applications. Any combination of such things can cause a loan application to be suspended, and potentially rejected.
With thousands of unique loan applicants, it is challenging to quickly evaluate each loan, and know what requirements must be met to return the loan into the approval process. Each unapproved loan represents disappointed customers and significant dollars in lost revenue to the mortgage company. Plus, every additional day a loan remains suspended, the likelihood it will fail to close increases.
Saffron worked with the customer to create a new Experienced-based decision support tool using SaffronMemoryBase and Saffron REST APIs. The solution helps company mortgage bankers rank suspended loans according to how likely they are to close, recalling similar loans successfully closed in the past to suggest modifications for the bankers’ consideration, and highlighting the shared characteristics of these similar prior loans.
Saffron developed the Reasoning Methods REST API for decision support, which includes rank- orderering loans most likely to close; finding similar loans; recommending actions; explaining how/why those actions will work; and automatically learning from actions and outcomes. The customer developed the end-user interface for the mortgage bankers, leveraging Saffron’s REST API for ease of integration.
As the banker works with a suspended loan, SaffronMemoryBase incrementally learns the outcomes – the successes and failures – of each of the banker’s modifications and incorporates these outcomes into SaffronMemoryBase for immediate use by all other bankers working on new and existing suspended loans.
Using Saffron now enables this banking customer to apply the mortgage department’s sum-total Experience across all loan decision makers, giving each the Experience of many others. Saffron also defined an extensive new “set” of more than 125 unique loan-status attributes as decision criteria, all unconstrained by rules-based or statistical business intelligence models.
The implementation is underway. Our goal is to improve suspended loan conversion rates by 3 – 5% annually. The net result is satisfied customers supported by better, faster loan decisions.
Making decisions is about weighing the likelihood of outcomes to select the most favorable action. To enable a better decision, a Decision Support tool needs to compute likelihood across scores of possible outcomes, and make recommendations based on past cases.
The basic premise of Saffron REST APIs is to add easy-to-configure Decision Support capabilities to SaffronMemoryBase.
Saffron models the essence of your Decision Support process with three simple concepts:
1. Cases – A case is an application entity upon which decisions will be made. In the suspended loan example, a loan is a case. The characteristics of a case are represented as a set of properties called attributes.
2. Attributes – An attribute is a name-value pair representing a property. The possible results of acting upon a case based on a decision are represented by a set of labels called outcomes. In the suspended loan example, the properties of the loan are mapped as the attributes of the case.
3. Outcomes – In the suspended loan example, the final loan status of approved or denied is modeled as outcomes.
The key functions for Saffron’s Decision Support REST API include:
Triage – The rank ordering of events/transactions according to their likelihood of outcomes. Rank ordering will be based upon the Customer’s defined factors which may be numerous and large in scale.
Recommend – Saffron will recommend the actions most likely to result in favorable outcome based on the case’s similarity to other case with favorable outcomes.
Nearest Neighbor – Saffron will identify existing cases within SaffronMemoryBase that are most similar to the case under examination. Similarity analysis is based on the distance between the attributes of a case at a given point in time or status and their values at an outcome.
Explain – Saffron will support the rank ordering and recommendations with two forms of explanation. First, prior similar cases and actions will be presented as the basis of prior experience. Second, the most discriminant factors for the rank and recommendation will highlight the most important aspects of similarity and difference.
Adapt – Saffron will adapt and learn over time by considering additional case experiences that occur when cases are resolved. Additional case experiences will be captured on the fly for continuous learning and improvement of the rankings and recommendations, providing a dynamic experience-based decision support capability
In Saffron’s world, Experience Management by Decision Support focuses primarily on Reasoning by Classifications and Temporal Analysis REST APIs. These allow you to reuse the Decision Support capabilities built into SaffronMemoryBase across the enterprise. You can also support a variety of operations and easily integrate Saffron into existing and new end-user interfaces.
By mapping the entities for a given situation into these simple concepts, and ingesting the data into SaffronMemoryBase, powerful Decision Support functions result, and they help you and your company make better decisions.
For more information about Experience Management by Decision Support, contact us.