This is a series of projects funded by the Centre for Defence Enterprise, Defence Science and Technology Laboratory of the UK Ministry of Defence over the period 2013-2018, in response to DSTL themed competitions. The projects were led by iARC.  

Themed Competition: Open-Source Big Data Insight - PHASE 2

• Automated Knowledge Discovery, Classification from Semi-structured Data Sources. Phase 2,  (Project ID: 7251)  2016-2018

Project Summary: The project extended the robustness, usability and functionality of the technology developed in Phase 1 (below).  The project delivered a set of software components which can be integrated into a larger framework allowing novel and complex analysis of vast data sets in near real-time.

Themed Competition: Open-Source Big Data Insight - PHASE 1

• Automated Knowledge Discovery, Classification from Semi-structured Data Sources. Phase 1, (Project: CDE 40385), 2015-2016 

Project Summary: The project developed a proof of concept to demonstrate modular technology that permits the ingestion of large quantities of data from a wide range of sources that can be either unstructured or semi-structured. The data is classified and ordered automatically using Probabilistic Topic Modelling in to topics and then processed with an unsupervised Deep Learning engine to identify individual anomalies (documents of interest) in the corpus. The Project developed fouropen source prototype modules: for  (a) Data Gathering and Image processing  (b) OCR (c) Probabilistic Topic Modelling and (d) Deep Learning.The project also developed a Dynamic Visualisation Module to both present search results and visualise trends and changes in the corpus. The work demonstrated was at TRL3.

Themed Competition: Information Processing and Sensemaking

• Auto-identification of emerging behavioural stereotypes from semi-structured data feeds. (Project: CDE36457), 2014 - 2015.

Project Summary:  This project devised and evaluated for DSTL an automated analysis system to dynamically enable the identification and classification of behavioural stereotypes from semi-structured data across multiple data representations, and predict future actions based on such behaviours. To achieve this the project used a two-stage analysis, an initial dimensional reduction followed by the use of Deep Learning to analyse complex hierarchical causality relationships, executing on GPGPUs.

Themed Competition: Cyber Defence - Securing Against the Insider Threat

• Cyber Defence: Identifying anomalous human behaviour in heterogeneous systems using beneficial intelligent software (Ben-ware) (Project: CDE34938), 2013-2014

Project Summary: The insider threat problem is a significant and ever present issue faced by any organisation. While security mechanisms can be put in place to reduce the chances of external agents gaining access to a system, either to steal assets or alter records, the issue is more complex in tackling insider threat. If an employee already has legitimate access rights to a system, it is much more difficult to prevent them from carrying out inappropriate acts, as it is hard to determine whether the acts are part of their official work or indeed malicious. This project has developed the concept of “Ben-ware”, a beneficial software system that uses low-level data collection from employees’ computers, along with Artificial Intelligence, to identify anomalous behaviour of insiders.