Except when we are asleep, our conscious brains are being bombarded with diverse data – structured and unstructured – and we process it for decision support.
The volume of diverse data is huge – our movements, and interactions with the world around us fed by all our senses into our brain. The process is almost transparent to us – we analyse it, we store it and we can retrieve it. Memories are continuous in form, too – almost like the Bayeaux Tapestry, though even that is limited to 230 feet in length.
We can look at and interpret the Bayeux Tapestry and even desynthesize it into structured data – dates, say, and unstructured data – the ‘feel’ of the tapestry itself, were we allowed to touch it.
Traditional systems cannot begin to behave in the way that our brain does.
The structured and unstructured data is seamlessly synthesized by our brains in a way that storing data in computers is not – at least until now.
The Tapestry is an extreme example, but if we take a relatively narrow and data-rich content – such as financial market performance then that is a useful place to start the quest.
Take the price performance of gold: we have structured data feeds from all the world’s markets, we have Reuters and other news feeds. We can build systems which able to parse the news feeds for the keywords like ‘gold’ and ‘precious metals’, say. With semantic analysers we can cross reference everything with a date.
We could even build in links to news stories with ‘crisis’ and ‘gold markets rise’. There, we have a tapestry, but it is in the sensemaking where systems traditionally fail.
The web of diverse data grows to vast for traditional BI to make sense of it.
As Forrester Research said, the use of associative memories is the new dawning.
That is the field that Saffron Technology is working in.
If we have enough gold tapestries which are continuously integrated forming an associative memory base, then we have the opportunity to use that for predictive analytics.
Though the volume of data is huge – just like the human brain, we can process all this diverse data and store the connections, counts and the context.
Then, without the need for rules, we can retrieve, analyze and look for patterns – to anticipate what might happen next.
