COGNIMAN Toolbox Portal

The COGNIMAN Toolbox is structured according to the COGNIMAN Reference Architecture, comprising a set of different Digital Twin (DT) supporting components. These component will typically be connected and configured together in different ways for different pipeline instances in various application contexts.

As shown in the figure, seven main steps for the pipeline have been defined, corresponding to 1) Physical Twin, 2) Data Layer, 3) Digital Twin Representation, 4) Service Layer, 5) User Interface, 6) Connectivity and Intergration, and 7) SSH and Ethics.

Toolbox

Digital Twin Pipelines

Defect detection in fibreglass production

Demonstrated through a pipeline implementation in a 3B Fiber plant.

Precision machining for deburring of large metal parts

Demonstrated through a pipeline implementation in a GOIMEK plant.

Additive manufacturing for medical implants

Demonstrated through a pipeline implementation in a CROOM plant.

Flexible manufacturing – Digital library for batches

Demonstrated through a pipeline implementation in an ABS plant.

Toolbox Components

Physical Twin

The physical twin represents the actual physical object, process, or system within the real world. This layer includes physical machinery, equipment, devices, and any other tangible components that are part of the manufacturing environment. Sensors and actuators embedded in these physical assets collect real-time data, which serves as the basis for creating and updating the digital twin.

Camera

Device capturing images (3B Pilot).

Sensors

Device capturing temperature, vibration, and tension (3B Pilot).

Deburring Tool Sensors

Triaxial accelerometer, force and torque sensor, 2D Gocator Laser, touch probe sensor (GOIMEK pilot)

External Sensor

External sensor located at height for detecting people and positioning the piece (GOIMEK pilot)

Spinnaker ROS2 Camera Driver for Blackfly Camera Sensors

AGV Camera sensor (GOIMEK pilot)

Livox ROS 2 Driver for Lidar Sensors

AGV Lidar sensor (GOIMEK pilot)

DepthAI ROS2 Camera Driver for OAK Camera Sensors

OAK Camera sensor (DepthAI ROS Integration)

Ouster ROS 2 Driver for Lidar Sensors

AGV Ouster sensor (GOIMEK pilot)

Video stream acquisition

The system must be able to acquire a video of the stock (ABS Pilot)

Data Layer

The data layer serves as the foundation for the entire digital twin ecosystem. It encompasses the collection, storage, and management of raw data generated by sensors and other data sources associated with the physical twin. This data can include measurements, sensor readings, operational parameters, environmental conditions, and more. Proper data processing and storage are crucial to ensure accurate and reliable digital twin representations.

Annotated Images

Annotated image dataset from EBD system (3B Pilot).

Break Rate Dataset

Timeseries dataset of breaks and relevant process data (3B Pilot).

Database Design for Manufacturing Data

The system provides a unified interface for interacting with different database backends (PostgreSQL, MySQL, SQLite) ...

Data Lake

Azure data lake for raw data storage.

Sensor Dataset

Timeseries dataset generated from sensors (3B Pilot).

Deburring Feedback Quality Model

Record of the feedback provided by the human on the quality of the deburring performed (GOIMEK pilot)

Deburring Trace Model

Record of how deburring has been performed in detail (GOIMEK pilot)

Map & Part Piece Model

Movement area map with points of interest, CAD model of the piece to debur (GOIMEK pilot)

Pilot Configuration Model

General configuration of machines, endpoints, addresses and user accounts (GOIMEK pilot)

Semantic Map Model

Semantic map with objects identified (GOIMEK pilot)

Digital Twin Representation

The heart of the architecture is the digital twin itself. This layer is a virtual representation of the physical twin, constructed using the real-time data collected from sensors and combined with relevant contextual information. The digital twin accurately mirrors the behaviour, characteristics, and conditions of its physical counterpart through modeling, such as Artificial Intelligence (AI), Graph based modeling, etc. It enables simulations, analysis, and monitoring, allowing for predictive insights and informed decision-making.

A DigitalTwin for Additive Manufacturing printing process

This project aims to develop a digital twin system for monitoring and optimizing additive manufacturing (AM) printing...

SINDIT

Knowledge Graph Based Digital Twin Framework.

Clearpath Robotic Simulator

Digital Twin Clearpath Robotic Simulator

Gazebo Robotic Simulator

Digital Twin Robotic Simulator

Real-Time ROS2 Board

Real-Time viewer of ROS2 in a web-based board

Real-Time ROS2 Lichtblick Visualizer

Real-Time viewer of ROS2 in a web-based fully customizable user interface

Robot process workflow simulation

Simulation of a Robot process workflow... (CROOM pilot)

Batch identification

The system must be able to acquire the batch 'code' at least in the 90% of cases (ABS Pilot)

Batch position

The system must be able to acquire: Ground clearance of the bar (y); Abscissa of the bar (X) (ABS Pilot)

Implementation of 2D map

Develop a 2D map that contains: Ground clearance of the bar (y); Abscissa of the bar (X) (ABS Pilot)

Position for different bar dimensions and sections

The system must be able to acquire batch position for different bar dimensions and sections (ABS Pilot)

Real-Time ROS2 Log Bag Player

Real-Time Log Player for ROS2

Service Layer

The service layer encapsulates the functionalities and services that leverage the capabilities of the digital twin. This layer includes various algorithms, models, simulations, and analytic that use the data from the digital twin to generate insights, predictions, and optimization strategies. It is responsible for translating raw data into actionable information and facilitating interactions between different layers of the architecture.

Break Detector and Characterization

Break detector models based on images dataset (3B Pilot).

Break Rate Analytics

Break rate models based on break rate dataset (3B Pilot).

Sensor Data Analytics

Service analyzing sensor data (3B Pilot).

Autonomous Deburring

Embedded software for autonomous deburring (GOIMEK pilot)

Autonomous Navigation

Embedded software for autonomous navigation (GOIMEK pilot)

Deburring Planning

Embedded software to plan deburring trajectories (GOIMEK pilot)

Global Mission Control System

Software to manage global mission and orchestrate subsystems (ARM, AGV, HMI) (GOIMEK pilot)

Navigation Planning

Embedded software to plan navigation trajectories (GOIMEK pilot)

Quality Deburring Feedback & Improvement

Software to improve next deburring considering human quality feedback (GOIMEK pilot)

Safety Awareness

Software to trigger events when people detected and position part piece precisely (GOIMEK pilot)

Semantic Map Generator

Software to recognize objects and create a semantic map (GOIMEK pilot)

BlackBox Service

BlackBox service for auditing

Automated Inspection

Automated inspection to be mapped and optimized processes... (CROOM pilot)

Automation of removing parts from build plate

A solution for automatically removing parts from build plate... (CROOM pilot)

Automation of support removal

A solution for automatically removing supports from parts... (CROOM pilot)

Automation of surface polishing

A solution for automatically polishing surfaces... (CROOM pilot)

Job Card Extractor

A Python tool for extracting job numbers and operations from manufacturing job cards (PDF documents) using OCR and ba...

Data Driven Design of support structures

Intuitive support generation based on geometry... (CROOM pilot)

Part handling and management

A solution for part handling (CROOM pilot)

Printing Process Monitoring

Monitor the quality of the printing on each layer... (CROOM pilot)

Devices certification

Devices must be certified for specific standards (ABS Pilot)

Frequency update of the 2D map

The frequency of update of the map must be at least once a day (ABS Pilot)

Integration with ABS custom warehouse management system

Integration to suggest batches to be picked (ABS Pilot)

Integration with ABS ERP (SAP)

The system must be able to interface with ABS ERP and provide ground clearance data (ABS Pilot)

Safety rules support (triangle)

The system must be able to recognize the shape of the bar (round), wedges, height, etc. (ABS Pilot)

Safety rules validation (square section)

The system must be able to recognize the shape of the bar (square), height, etc. (ABS Pilot)

SAP custom transactions (ZNET e ZBATCH) adaptation

The system must integrate the ground clearance data in ABS ERP (ABS Pilot)

User Interface

The user interface layer provides a means for human interaction with the digital twin ecosystem. It includes interfaces such as graphical user interfaces (GUIs), dashboards, augmented reality (AR) or virtual reality (VR) applications, and other visualization tools. This layer allows users, including operators, engineers, and managers, to monitor the state of the digital twin, access insights, and make informed decisions.

Connectivity and Integration

The connectivity and integration layer ensures seamless communication between the various components of the architecture. This layer facilitates data exchange between the physical twin, data layer, digital twin, service layer, user interface, and external systems. It might involve technologies such as Internet of Things (IoT) protocols, APIs (Application Programming Interfaces), middleware, and networking solutions to enable smooth and reliable data flow.

SSH and Ethics

The Social Science & Humanities and ethics layer focuses on the responsible and ethical use of digital twin technology including social aspects. As digital twins gather extensive data, including potentially sensitive information, ethical considerations around data privacy, security, consent, and transparency become paramount. This layer addresses ethical concerns and ensures that the implementation of digital twins respects legal and societal norms, and safeguards data and privacy.

Face Blurring

A modular face anonymization system for ROS2