The purpose of the Laboratory Integration and Testing #1 (LIT#1) was for the end users to test available functionalities of the INGENIOUS Augmented Reality Services. Due to the COVID-19 situation, a virtual meeting was set-up in October 2020 and remote demonstrations were performed with the device screen being shared and pre-recoded videos being played showing the available features in use. During this LIT, our partners from CS-GROUP introduced the Augmented Reality (AR) to the end users and what an AR application is. They presented, then, the selected AR devices (ORA-2 and HoloLens 2), the capabilities offered by each, and the considered roadmap for the release of the features. The demonstrations highlighted the key features relevant for the First Responder (FR) uses cases:

• What would be an AR notification (An alert related to an imminent or nearby threat or a command from the control centre).
• What a hologram looks like (3D model, image or a text visible in the environment) and what it would be useful for (share the location of a victim or of a risk, give visible information and free),
• The types of interactions the FRs can have with the application (using voice command or holographic buttons).

The discussion was fruitful and feedbacks very positives. End users proposed to include useful information in AR such as the current date time and the capability to start a count down. For the holograms, they proposed to use the ADR sign to identify hazardous substances and to add the capability to mark doors and buildings. Finally, the need for direct access to what the FR sees into the control room was expressed.

LIT#1 included also functional testing related to video and voice communication on the 1st prototype of the INGENIOUS helmet, which is being developed by our partners from KIRO. This prototype is equipped with EO/IR cameras and a wireless voice communication module that are integrated into the helmet. For the testing, a two-story smoke-filled test building was set up and a victim (thermal mannequin) was placed in the second-floor room. The demonstration was based on the search for the victim and a video/voice communication in a two-story environment full of smoke. The results of the self-evaluation are as follows:
  • Image loss Occasional occurrence and EO/IR Sink Slightly Off, need to improve video quality
  • Good audio communication but there is a need to be able to connect with various voice communication devices

In LIT#1, end user opinions were collected for the development of the 2nd prototype. As a result, we have established a development direction to mount detachable electronic equipment on commercial helmets by reflecting the demand of FRs for helmets that are strong enough to meet EU standards.

2nd round of LITs

LIT#10

Like LIT#1 of the 1st round of the INGENIOUS LITs, LIT#10 of the 2nd round of LITs was performed remotely in April 2021 and the purpose was three-fold. First, to summarize the outcomes of the previous LIT; what was presented and the gathered feedbacks and how they have been addressed. Then, to present the features that were being researched and developed up to this LIT. Finally, to present the roadmap to the future Small-Scale Tests (SSTs), and what will be proposed in the successive versions, two and three, of the AR prototypes. Thus, CS-GROUP presented the current axis of R&D and the latest status of the components:

• What would be the non-persistent AR menu displayed on the hand. The goal is to give access to more actions without overloading the FR field of view.
• The latest progress regarding the map with FR and hologram geo position.
• The envisioned holograms types and their representation. Created from the COP (on the map) or from other FR in the fields. Current types: risks (electrical, fall, biological…), victim, switch (electrical, gas, water), incident, and marking on door/building.
• What and how triage information will be presented in AR. Using the AR glasses camera to scan hand free the triage device QR Code. 
• The remote assistance component and what would be possible to do with it (Access FR video stream, hand free annotate the FR view…).

End users were enthusiastic and looked forward to testing the AR tools during the SST. Amongst other comments, participants proposed the use of non-realistic Holograms to avoid confusion as the hazardous signs presented were too realistic. They requested the use of standard triage form and they were wondering about the voice commands robustness (to noise, to multiple FR with AR device close to each other). Our partners from CS-GROUP ended up the presentation with the plan for the SST/version 2 prototype (notifications coming from the fusion engine, annotations shared between the AR user and the COP, mission from the COP, triage AR application), and the envisioned features for the version 3 (extended annotation, timer, remote assistance).

In LIT#10, KIRO also showed a live demo of the 2nd prototype of the helmet that reflected the end user’s request. Not all modules were developed yet, only the front EO/IR camera module has been tested. The testing scenario was a situation in which the building was full of smoke due to gas leakage. The test environment was almost the same as last LIT#1, the victim was somewhere on the second floor in a two-story smoke-filled building and the goal was to find the victim. User feedback on the results of the demonstration is as follows.
  • There is no status LED and turn ON/OFF button
  • What about the replacement batteries
  • How to handle smoke problem with an IR camera

In SST#1, KIRO plans to complete the headset for voice communication, detachable battery, and computing modules under development and reflect user requirements.