On the battlefield the challenges the military faces change very, very quickly. During the wars in Iraq and Afghanistan there was a push from the Pentagon to digitize the battlefield, including using real time analytics. Dr. Noel Greis, Director at the Center for Logistics and Digital Strategy at University of North Carolina, explains how an innovative collaboration involving Saffron was able to help Boeing manage the battlefield supply chain using real time data analytics.
The supply chain is an essential, complex and very costly element of any war zone. Loss or delay of materials or assets could seriously impact a conflict situation. Nevertheless, in the past, supply chains have been managed in a relatively straightforward manner. Soldiers monitor inventory, and request resupply when levels run low.
The advent of the digital, sensored battlefield presented a new opportunity to manage the supply chain more efficiently, using real time analytics to help improve decision making.
Boeing has a very large, defense systems division, and works with the Department of Defense. Boeing contacted Dr Greis at the Center to see if she could help them develop some new tools that will enable them to manage the battlefield supply chain, in real time. A collaboration was put together including a company that did battlefield sensors, and the idea was to sensor up the supply chain.
The supply chain included platforms in the battlefield (e.g. tanks or supply trucks). So think of the battlefield as a huge sensor field with all the vehicles sensored.
Inventories are also sensored.
Then you can now take all of that information and bring it into an information space.
Saffron was then used as a data analytics tool to make sense out of that information, making recommendations about how to allocate the inventories.
One asset which is invaluable is fuel. With the sensors able to track when fuel was needed, and the delivery location, Saffron’s real time analytics are able to make more effective recommendations.
Although the system was designed to be autonomous, there are options for a human to be in the loop, so that if Saffron recommended re-supply on a certain date, of a certain amount, at a certain time to a certain set of vehicles, the commander could okay it or not okay it.
The commander would still be part of the decision making process, but it made fuel delivery more efficient, more accurate, and involving less guess work.
The implications in a commercial sense are obvious.
With greater information awareness along the supply chain about inventories, companies in the logistics sector could be more cost effective.
Anything which currently relies on human judgement for determining when something is needed could be supported by real time analytics, which would verify when materials, goods, and assets are actually required.
The same efficiencies Boeing sought on the battlefield with Saffron will have commercial implications globally, as business embraces Big Data.
Dr. Noel Greis, is the Director of the Center for Logistics and Digital Strategy, University of North Carolina, and is an executive advisor to Saffron.
“The human mind … operates by association. Selection by association, rather than indexing, may yet be mechanized.
Vannevar Bush, As We May Think, 1945″
On 22 February I had the privilege to speak at Raytheon’s National Innovation Day about Saffron’s Associative Memory technology, as an example of innovation, invention, and even revolution. This was a special moment for me, because of the amazing connections between Raytheon’s founder, Vannevar Bush and Associative Memories, which is my life’s work.
Bush is widely regarded as the father of Modern American Science and is considered the greatest American engineer of the 20th Century. He organized the Manhattan Project. As Director of the Office of Scientific Research and Development, Bush led six thousand top American scientists in the application of science to warfare. He was also instrumental in the founding of the National Science Foundation (for which I am personally thankful as a NSF Graduate Fellow, helping me to get through grad school).
And somewhat incredibly, back at the very birth of computing, he believed that computers should operate by association rather than indexing. In his famous paper in the Atlantic Monthly, “As We May Think,” Bush described a memory-based assistant, what he called a MEMEX, that would read and remember everything to assist our own memories of everything we know. He worked on the MEMEX for decades of his life, but with only microfilm to work with, he was too far ahead of his time. Ultimately, he is credited with the inspiration for hypertext, linking documents together as we now enjoy of the World Wide Web. However, Bush meant to link ideas within documents, which we now see in the rise of the Semantic Web about “things” in data and how they are connected.
Personally, I think that Bush meant even more in his thinking about associations. While he was engaged in the birth of computing in general, he was more interested in analog rather than digital computing. For Saffron at least, we define “associations” as both the connections of a graph as well as the counts or frequencies, the analog “oomph” of these connections. If you think about neural connections as the basis for how we think, it’s all connections and counts, synapses and strengths. What else is there? There is a lot more to how we think, of course, but these are the fundamental elements: Connections that underpin semantics and counts that underpin statistics. In the same way that 1s and 0s became the basis for the universe of apps on your smart phone and all of digital computing, connections and counts are the elements for more cognitive, intelligent computing and all that can be made of them.
Fast forward to 2012 and we find ourselves at the ‘right time’. Key analysts like Forrester & Gartner, as well as private enterprise, have woken up to the Associative approach. In both National Security and in certain large commercial innovators, Saffron has been applying Associative Memories for sense-making and decision support over the last decade. What drove the early adopters was the arrival of Big Data as both a problem and opportunity. This is perhaps best illustrated by national intelligence efforts to identify terrorists in an avalanche of data since September 11 2001. Ten years later Big Data is a problem and opportunity for business at large. The chaos in the economy combined with the failure of traditional BI techniques — at least the disappoint of “spotty success” if not abject failure — make change inevitable. As I described the nature of revolutionary change at Raytheon’s Innovation Day, new market needs and political needs inevitably emerge and will be inevitably met by a new way of thinking.
“Every revolutionary idea seems to evoke three stages of reaction. They may be summed up by the phrases:
1- It’s completely impossible.
2- It’s possible, but it’s not worth doing.
3- I said it was a good idea all along.”
Arthur C. Clark
Data Driven decision making requires a new BI
Whenever I hear “business intelligence” or “consumer intelligence” or “organization intelligence” or “national intelligence” I think of the human brain. A brain of neural associators and synaptic connections that is truly intelligent. THIS is how we think, how we make sense and make decisions in the real world. And this is how Saffron thinks. Connections and counts in context. What else is there?
So I sign off with these comments from my colleague Paul Hofmann. In this video he gives four reasons why the future lies with Associative Memories, just as Vannevar Bush predicted in 1945.
In this two part article, Dr Noel Greis describes an application using the Saffron Sensemaking Engine and battlefield sensors to help make real-time decisions and manage transactions in the military supply chain.
In the battlefield it’s very important to have what the US military calls Situational Awareness.
When you’re aware of the situation, you’re aware of the resources that you have, the resources that are needed and anything that would describe the current situation or affect the need for resources.
Around the time of the Iraq war the Bush administration placed a very strong emphasis on bringing technology onto the battlefield. The US sought a more networked, information-rich battlefield environment.
A key aspect of that effort was having the right resources in the right place at the right time.
Making sense of an organic, ad hoc, real time process.
“Johnny, I’m really running low… you’ve got to get X to me and you’ve got to get it to me in the next three hours!”
This is a typical supply chain conversation. Reactive, and dependant on people interpreting events as they notice them. A management approach that gets less effective as the complexity of the supply chain increases and likelihood of unforeseen events goes up.
In our application Boeing had a classic resupply mission – whether it was resupply of water, of gasoline, of ammunition, anything that might be needed.
The idea was to gather information in real time from the battlefield and use Saffron to make decisions to do with managing the transactions in the supply chain.
The world is becoming sensored.
Whether it’s from vehicles that are in the battlefield, from aircraft above the battlefield, or from the actual humans that are in the field, we can bring all of that information together and make decisions about how to respond.
The battlefield is an increasingly sensored environment. You have sensors on all the vehicles. You may have sensors on the soldiers. You have sensors everywhere. The application gathered information from all of the platforms and all the individuals.
We created a system where we had situational awareness of everything that was happening in the battlefield as well as along with the supply chain.
We pushed all this data into our Saffron engine within a decision support environment to make recommendations about how to launch a resupply mission, when to launch it and what route to take.
You’re using Saffron to not only observe but to make decisions. In this case decisions about how to resupply from forward positioning stations or from a port or from any place where you would draw assets.
Many technologies came together in this particular application. Saffron was the core engine that we used to interpret the information. But we had to have other technologies that would that would bring data from the various sources – from the vehicles, from the individuals, networking software, and then software that enabled information-sharing across all these elements.
I recently reread “Extreme Risk Management. Evolutionary Approaches to Evaluating and Measuring Risk” by Christina Ray.
It’s a great read and one I highly recommend.
Christina believes that the intelligence community and financial communities face essentially the same issues when it comes to anticipating and managing the risk of extreme events (think collapsing currencies, sovereign defaults, bank runs etc).
She argues that the financial industry should be using the technologies adopted in National Security. By using real time market intelligence (“MARKINT”) as part of an all source intelligence strategy, risk managers can make sense of volatile markets in the present and better anticipate extreme events. Rapidly changing, market dynamics combined with an increasing number of complex finanical instruments challenges traditional historically based prediction models to perform accurately. We argue that new approaches are needed to generate actionable intel ahead of the financial equivalent of a terrorist threat.
From Saffron’s direct experience in National Security, I agree with Christina that the Financial Industry is not yet fully aware of just how far sense making, MARKINT style sensemaking technology has come.
Persistent, extreme risk is our new business landscape, and may well be for the foreseeable future.
It is the ability to unify numerous structured and unstructured data sources in real time, combined with a sense making tool and analytic reasoning methods that learn about, associate and connect new events with your interests – also in real time – that allows decision makers to anticipate and manage the risks around future black swan events.
People are looking for the edge, where is the information that I can leverage to get ahead of the rest of the market? Unstructured data seems to be the untapped source, whether it is news, social or both.
The Financial industry clearly calls out for a new approach to predictive analysis or what Saffron refers to as anticipation or “anticipating the future”. Prediction in this industry has been characterized by the use of a massive amount of historical market data reduced to models about the past to create forecasts about the future. But when the present is characterized by the increasing complexity of financial instruments and the increasing volume, variability and velocity of available data, then models based on the past become inadequate, uninformed and misguided.
Some news organizations, like Reuters and their work with OpenCalais, have been trying to help by bringing unstructured text into this analysis. OpenCalais automatically “reads” Reuters content and marks up the people, places and things (entities) that are being written about in the news article.
Dow Jones takes a similar approach to entity markup and makes these details available to algorithmic traders for analysis. In the past, accurately representing the people, places and things in data has been a big challenge.
But the question remains, even after the markups are done, how do we capture all of this knowledge and better exploit it to inform decision-making and help us anticipate the future?
To this end, change is underway on a variety of fronts. Take the recent sentiment analysis of Twitter and the news to help anticipate socio-political change. By analyzing public sentiment, as expressed in text, you can identify changes in public opinion or attitude. Arab Spring is a good example:
A new study by Computer scientist Kalev Leetaru, at the University of Illinois at Urbana-Champaign, claims you could have foreseen the Arab Spring if only you’d been paying enough attention to the news. Leetaru shows how data mining of news reportage can reveal the possibility of future crises well before they happen. Leetaru mined news reporting to identify the sentiment, as reported, on Tunisia, Egypt and Libya. “In all cases”, he says, “there was a clear, steady trend towards a negative tone for about a decade before the revolts.”
Monday morning “quarterbacking” is always easier. Leetaru knew that long-established dictatorial political leaders had been deposed. In other words, he knew what to look for in this case. Despite that, his argument is that sentiment analysis can be an indicator of an impending event, crisis or opportunity.
The ability to analyze unstructured data sources is an important component of analysis, such as anticipating socio-political unrest or mitigating brand reputation risk or determining the best approach to replacing damaged non-inventoried components. Incorporating unstructured data sources in business decision-making is proving to be a viable component of business intelligence. It provides context, facts and additional insight about a topic or transaction that cannot be expressed by predefined structured data.
The automatic combination of text or unstructured data with structured data creates a new, unified knowledge source. This marriage of structured data with unstructured sources not only changes the way analysis is done but creates a more informed, more complete and more accurate knowledge base to exploit for decision making and anticipation.
Without this, text analysis systems are just saying that the overall sentiment toward Egypt is bad (and you know this by reading and looking at the overall trend).
But a deeper understanding of the current (and past) relationship of people, places, things and events makes anticipating potential outcomes possible and thereby creates actionable, informed intelligence for risk management, reputation management, and a variety of business decisions.
In our personal finances, we want to know “should I invest in this stock or not? Should I divest of this stock or not? Should I stay in equities or move to cash?”
Elevating business intelligence to answer such questions cannot be accomplished by just analyzing unstructured text sources.
At Saffron we’ve created a new school of business intelligence that combines text sources with more traditional structured, transactional data sources. We do this through a specialized data index based on associations. Saffron stores the semantic structure of your data, along with frequency counts so that relevance metrics can be computed. Now you have an agile knowledge source to exploit, in real time, for decision support and anticipation applications. You can now answer questions such as “which of our customers will churn given a competitors new product announcement? Who are these customers? Why will they churn? What has happened to provoke churn?”, without predefined models or rules, and instead using all the information available about a customer, your interactions with them, all of their transactions with you and ultimately their interests and concerns as expressed in their communications with you.
Understanding how and why markets and customers move as supported by a variety of real time and historical data about indicators, transactions, socio-political events, and public sentiment is the Holy Grail of Analytics. Bringing all this together into a naturally predictive system is the Holy Grail of the Financial Industry.
On second thought, isn’t this the Holy Grail for all industries?
You have to first understand that there is a plethora of information locked away in unstructured sources. The ability to get to that data is somewhat convoluted.
Saffron’s experience with national security pre-9/11 has highlighted this ‘big data’ problem particularly with data coming from news, email and many other sources of unstructured content. Saffron figured out that you could use plenty of tools, called ‘entity extractors’, as one element of dealing with text. So if you read documents and comments it is not just about storing the text or data, but about reading it in some sense of finding what:
What things are?
What are the parts?
What are the problem descriptions?
Who are the people?
What are the companies mentioned?
What products are mentioned?
When you start using these ‘entity extractors’ to find, and mark, the people, places and things the situations, problems, actions and outcomes give Saffron a memory base. This leads to opportunities to learn about those things and connections between them, the connections between people and the connections between problems and solutions.
It’s in the text and part of the solution to address text is first to use these technologies of marking people, places and things, situations, actions and outcomes. If you mark these then Saffron as a memory base can help you learn about all those people, places and things, situations, actions and outcomes and how they are connected. That’s the knowledge, and the connectivity of the data. It is not just the raw data itself and this is how Saffron gets into the unstructured text to exploit it.
Natural approach sees 10 to 20 times return on mission.
When people make decisions, we recall our past experiences to find the analogies and similarities. What was like this situation? What was different? What did I do?
How did it turn out?
This is the way the world works… except in business where increasing focus on data has meant more rule based decision-making.
“Less than 20% of information we need to make decisions is stored in the company’s data warehouses. A new approach is needed… DBMS are not capable of handling and searching through all the types of data we have – email, text, voice, video… ” – Donald Feinberg, Gartner VP
High Velocity, High Complexity, High Diversity
A fuzzy world of diverse, complex and disorganized data sources can’t accurately be explained via tables and SQL queries. Indeed, 80% of the answers can’t even be found within your business.
Big Data is not just big in volume but big in velocity as intelligence goes real time.
According to Forrester – current BI technologies are “falling short in the ever-faster race” to meet business needs. (Download report free)
Unlocking experience unlocks ROI
Like natural human intelligence, Saffron focuses on how People, Places and Things fit together.
Like a true corporate memory, it doesn’t require rules, models or statisticians. You don’t even have to decide what to include or exclude. Everything you can know is included.
One global manufacturer has experienced a 10x return on cost of ownership versus savings in year 1.
And because there are always new problems to solve, this is a recurring business case. In year 2, ROI is expected to grow to 20x.
Democratized and operationalized.
BI is no longer just for forecasting, or just for management. Democratized data means real time intelligence within your company. Available to anyone and everyone as needed to get the job done.
Saffron Advantage: experience intelligence without rules.
While applicable in virtually any business environment, Sense Making is well known for its value in military Intelligence analysis. Saffron has first-hand experience applying this dimension of our robust data analytics solution to the problem of IED (Improvised Explosive Device) defeat in Iraq.
Sense Making Objective
Draw on Experience to save lives by finding enemy insurgents and stopping random bombings before they occur.
The U.S. Army and its coalition partner war-fighters in Iraq needed the ability to decide and act at the tactical level to eliminate, capture or exploit insurgent targets directly or indirectly involved in IED activity. To be effective, this targeting had to be done prior to actual IED attacks (and at the time of our forces’ choosing) in order to prevent loss of life for both civilians and war-fighters.
Saffron Technology played a lead role on the team formed to enable and enhance the analytic workflow of war-fighting intelligence analysts. SaffronAnalyst was developed by Saffron to provide an analyst user interface consistent with the workflow and thought processes of an actual Intelligence analyst. Although the system was already capable of identifying associations and their frequencies between basic entities like people, places and things — Saffron was asked to provide this analysis within the existing analyst workflow by integrating Saffron’s Sense Making capabilities with other third- party tools to provide an end-to-end analytic workbench.
The Customer-Saffron Solution
We worked closely with military Intelligence analysts, and Saffron’s engineers were trained in all the current methods of analysis used by the customer. The customer’s existing processes involved manually extracting entities and building a co-occurrence matrix. Saffron identified immediate opportunities where the Saffron Natural Intelligence Platform could improve the “time and speed to results” over their current manual process.
Saffron engineers observed Intelligence analysts in action and spent time on-site witnessing their analytical process in real time, against real targets. This made it easier to identify where SaffronAnalyst should integrate into the analysts’ workflow, so as to enhance — but not disrupt — current procedures.
The solution called for adding new features and capabilities to Saffron’s existing product to incorporate the desired components of the current manual process. At the same time, the customer created new approaches to parsing unstructured data. To ensure a streamlined and accelerated data-analysis process for the customer’s smaller operational units, Saffron and the customer teamed with a third-party ETL (data Extract, Transform and Load ) vendor.
Integration of the two products was developed, tested and implemented in under four weeks. Intelligence analysts reduced the time for creating their Critical Entity Link Charts by 80% in the final benchmark test. And Saffron further enhanced its Sense Making tool.
Many organizations face a similar challenge of quickly analyzing large volumes of intelligence data with a small team of people. Saffron Natural Intelligence Platform’s Sense Making capabilities make it ideal for helping organizations and their workgroups analyze large volumes of such data in a short amount of time.
Instead of two-dimensional models and manually designed tables, Saffron uses dynamic, multi-dimensional memories to quickly identify connections and their frequencies of occurrence within massively dense and “noisy” data sets.
In this particular case, Saffron was integrated with entity extraction tools including SRI, Inc.’s NetOwl and Basis Technology, Inc.’s Rosette, and ingested varied unstructured data sources into SaffronMemory Base. Saffron’s analytic results were then integrated with i2, Inc.’s Analyst Notebook to provide complete transportability of the Connections, Analogies and other supporting evidence from Saffron directly to the Intelligence analyst’s desktops. In addition, new visualization capabilities for SaffronAnalyst were added, including new link analysis capabilities, entity management, export to ESRI’s ArcGIS product, analyst interest collaboration, and more. Throughout, Saffron’s Connections, Analogies and Network Reasoning Methods were used in this Sense Making implementation.
The value emerges in the extraordinary speed and accuracy with which the Saffron Natural Intelligence Platform returned its data analysis, giving Intelligence analysts enough confidence to act, and enough time to save lives.
Experience Management by Sense Making is Saffron’s proven, sophisticated method of fast, comprehensive and pertinent data analysis.