Open-source Text Data Intelligence System Based on the Domestic Visual Programming Environment PolyAnalyst

Abstract

The article is devoted to the application of natural language processing (NLP) technologies to enhance open-source intelligence (OSINT) analysis in government agencies responsible for ensuring national security. With the ever-increasing data flow from various online sources, efficient information processing and analysis is critical for timely detection and prevention of threats. To solve this task, advanced machine learning and artificial intelligence algorithmsare implemented using low-code principles. This allows significant acceleration of the development and implementation of analytical solutions, minimizing the need for traditional programming and involving specialists with less programming experience, but having expertise in data analysis and OSINT. The central part of the system under consideration is the PolyAnalyst visual programming platform. Its architecture, based on low-code principles, provides users with an intuitive interface for building complex analytical pipelines. The system includes specialized sets of functional nodes designed to perform various stages of data processing. The ETL (Extract, Transform, Load) module provides loading, cleaning and transforming data from various sources. The DataMining module enables the application of machine learning algorithms for pattern discovery and prediction. A key element is the TextMining module, using NLP algorithms to analyze textual information, extract keywords, determine sentiment, and identify entities of interest for OSINT analysis. Finally, the BI (Business Intelligence) module provides analysis resultvisualization and builds interactive reports for easy perception and decision making. Altogether, these modules enable the creation of powerful and flexible analytical systems that can process large volumes of OSINT data efficiently and provide valuable information for strategic national security decision-making.