Perceptual and Cognitive Systems

2021-03-01 – 2025-02-28

HAVATAR-Robots

Healthcare Added-Value Applications of Tele-Autonomous Robots

With the steady ageing of the world population, more people are confined to live in isolation at their homes. Also, health and environmental concerns are demanding people work remotely more often to limit the carbon footprint in travelling and reduce the risk of spread of viral and infectious diseases. In both cases, the contact with familiars, friends or coworkers rely on telecommunication solutions with low interaction possibilities and unsatisfactory social bonding. Such solutions lack social presence and spatial accessibility, as well as diminished body language and implicit communication cues. The HAVATAR project proposes to increase the level of autonomy and social presence of current systems by using recent robotics and artificial intelligence technologies. Further to the interaction, telepresence robots can play active roles in providing help and health assistance to quarantined people. Specific behaviors, actuators and sensors can play important roles in handling epidemic crises. Although focused on health applications, the technology developed in this project will have an impact in many other uses of telepresence robots that will improve the lives of the homebound: in education at distance, accessibility to cultural spaces and events, attendance to professional meetings and scientific conferences, among others.

2019-10-01 – 2022-09-30

FACING

FACe photo understandING

The main objectives of this project are to conduct exhaustive batteries of tests of tools created by INCM for the manipulation of images of people's faces for identification and travel documents. The challenges to be overcome include: (1) the processing of images from vision systems consisting of medium and medium-low quality and resolution cameras, namely current vision systems already in operation in enrollment portals and for which it is intended to increase the lifetime, maintaining or improving performance, including the removal of the background of the images, allowing an intelligent segmentation of the pixels relating to the photographed person (foreground) and pixels relating to the background (background); (2) analysis and verification of compliance with the requirements and recommendations of international organizations on the photographs of people contained in these documents; (3) facial recognition through comparison between feature vectors of faces stored in a Database and feature vectors of a person presented for authentication, either in the context of validation (1:1) or identification (1:n) and (4) proof of life to distinguish between the face of a living person and fraudulent objects or artifacts that intend to imitate a living person. The algorithms will be implemented by INCM, integrating technology and algorithms from ISR-UC. Exhaustive tests will be carried out to measure the defined KPIs, as a form of quality control of the algorithms.

2022-07-01 – 2025-12-31

FACING 2

FACe photo understandING

The FACING-2 project aims to study and develop methods that allow exploring facial biometrics of human beings in a safe, stable way and that complies with national and international privacy and security regulations. These methods will then be used in the construction of complete biometric systems (end-to-end), respecting the same security and privacy requirements. The project will explore the following main axes: compliance of portraits of ID documents with ICAO/EU requirements, facial recognition, liveness detection, morphing design, biometric template protection, face image quality assessment (FIQA), and user interface that allow the construction of apps with a modern and friendly usability. Regarding the techniques and methodologies used, the FACING-2 project will have a strong component of artificial intelligence and machine learning in computer vision, using the latest and sophisticated algorithms according to scientific literature.It should also be noted that the FACING-2 project will have a strong engineering component, since it is intended to industrialize the technology assuming realistic scenarios of product development. Specifically, some use cases are foreseen in the project technology. On the one hand, an intended use case is the self-registration of citizens at a distance in government systems and portals, allowing citizens, for example, to request the renewal of their civil identification or distance travel documents, without the need to travel to the official premises of the respective public authorities, reuse of your biometric data stored in a centralized system. On the other hand, a second use case is linked to the possibility of a person authenticating to a physical system, either using an ID or travel document, or using only his face as a biometric guarantee of physical integrity.

2025-05-01 – 2028-04-30

BCI4ALL

Brain-computer interfaces for people in Locked-in state and with disorders of consciousness

Project BCI4ALL aims to research and develop effective EEG-based BCIs for individuals in locked-in state (LIS) and completely LIS (CLIS). The focus is on machine learning methods to improve BCI usability, the use of tDCS for neuromodulation, and LLM-driven communication. It is also expected that researched techniques have the potential to be used for assessing patients with disorders of consciousness (DoC).

2025-03-01 – 2026-01-31

BlockDFake

Deepfake detection in videos and visual media content certification for the Public Administration

<p>The proliferation of generative AI, particularly image and video manipulation, which we call deepfake generation, allows the creation of videos with misleading content, showing persons in false contexts, even making declarations that never happened, or were never recorded. The potential for this technology for misdeeds, illegal usage or systematic population control is not recent, with examples of such usages in elections, political and social discourse (recall the recent case where a teenager in Portugal used these technologies to expose fake pornographic videos of school colleagues, also teenagers).</p><p>This technology may also be used for ludic, marketing and commercial means, with clear societal benefits. However, it also presents serious risks for society when used for illicit means. This project aims at mitigating the risks of illegal deepfake usage by creating mechanisms to detect such usage, but also by allowing content creators to employ a signature to mark their content as legitimate. The main idea is to create algorithms that complement both approaches allowing to protect image and video content in the public administration sphere. The image manipulation problem may be approached either by preemptively stopping or detecting it. On the detection side, there is the possibility of creating algorithms to detect fake content, such as morphing and presentation attacks. In this case, these algorithms may be adapted to video, even though the literature shows that this approach is not enough to handle high-quality deepfakes.<br></p><p>Our goal is to use implicit neural representations. We expect that the mathematical properties of said formalisms will allow the video classification based on the inconsistencies of visual content in said representations. The main advantage of this approach is that any manipulations may not be obvious in the pixel space, but may be highlighted by converting it to a continuous function, as pixel-space manipulations are sophisticated enough that it consistently fools human scrutiny. On the other hand, the deepfake problem may be approached preemptively, by marking the frames using robust steganography methods that allow validation by the viewer with a smartphone. In this case, the approach would be based on the expertise of ISR-Coimbra regarding steganography, which will be adapted to video.<br></p><p>The work plan will be oriented to algorithms and in the two approaches mentioned above. Additionally, a second line of work would be the identification of an architecture of a validation and video protection system or service, storing digital content signatures, or deploying protection models as Software as a Service (SaaS). The goal of this architecture design is mainly to pave the way for a future framework to validate visual content in the Public Administration that, on one hand, would allow public institutions to authenticate and certify any image or video content and, on the other hand, this architecture would allow for the fulfillment of compliance rules of AI Act, by allowing content creators to disclose whether the content is real (not generated) or generated by AI systems.<br></p>

2025-03-01 – 2026-01-31

CertifyMeAP

Professional Training Certificate at the Portuguese Public Administration: security, availability and reliability

<p>The INA (National Institute of Administration I.P.) is the public sector entity that has the prerogative of coordinating the information on Professional Training, with a special focus on training for Public Administration (AP) workers. The training courses provided by INA are of great importance for AP&#8217;s technical staff and senior management, which is why Professional Training Certificates are desired and, potentially, highly falsified elements. INA currently issues paper certificates with a classic security level, with room for improvement. This project aims to solve three main vectors: security, reliability and availability. On the one hand, the security of Certificates can be improved to a very high level and their contents comply with strict criteria that allow the integrity of their content to be verified [security and reliability]. On the other hand, it is intended that Certificates can take a qualitative leap in terms of ease and convenience of access, making them digital, through a digital twin, and with a more qualified digital management system and to which several digital layers can be added [availability].</p><p>When it comes to digital layers, one of the most important aspects of Certificates is the possibility of them being digitally signed, with digital verification, but also on printed documents, making the signature verifiable, but invisible (implementing printer-proof steganography technologies, also developed by the ISR-Coimbra team). Finally, the creation of a digital twin, allowing in some cases its total, or almost total, dematerialization, may also allow adding layers of related services. The Certificate can be made available by INA through an online platform with an access code (for instance, following the model of the permanent online records of the Institute of Registries and Notaries&#8211; IRN).<br></p><p>The use of AI is intrinsically related to the project&#8217;s main scientific challenges. On the one hand, the integration of the physical elements of the Certificates, detected and validated through a mobile device with a camera, brings demanding challenges in terms of Computer Vision algorithms. The solution will largely be achieved through the use of Deep Learning. On the other hand, the digital layers related to signatures with asymmetric encryption, related to natural language processing with large language models, bring great challenges to the project whose solution is also achieved through artificial intelligence (note that one of the requisites is to secure the whole textual information of the Certificates). The idea of the project is to build a proof of concept for both the physical certificate and its digital twin.<br></p>

2025-02-13 – 2026-08-12

UpReGain

Gamified brain-machine interfaces for personalized upper-limb post-stroke rehabilitation

Stroke is a leading cause of disability and the second leading cause of death worldwide, affecting millions of people each year, and only 15% of patients fully recover and return to normal life. The UpReGain project uses a BCI-based rehabilitation system to enhance the effectiveness of motor rehabilitation. Additionally, it aims to automate the rehabilitation process for home use and enable telerehabilitation. The BCI system is designed to detect patients' movement intentions, which will then be executed by either a robotic hand exoskeleton or a virtual hand. The rehabilitation tasks will be gamified, as gamification has enhanced patient motivation and engagement, thereby facilitating motor recovery. This research project is a collaboration between the Institute of Systems and Robotics - University of Coimbra (ISR-UC) and Coimbra Local Health Unit (Unidade Local de Saúde de Coimbra - ULS).

2025-03-18 – 2026-03-02

DeepFakesFLY

DeepFake detection made faster

This project develops DeepFake detection by training AI on a dataset of bona fide and mala fide content encoded via neural implicit representations. Preliminary studies suggest reconstruction errors differ significantly between real and manipulated videos, enabling a novel detection algorithm. High-performance computing (HPC) is critical to accelerate SIREN-based image reconstruction and metric analysis, which would otherwise take years instead of months

2024-11-01 – 2028-10-31

ACHILLES

Human-Centred Machine Learning: Lighter, Clearer, Safer

<p>ACHILLES delivers a modular framework for developing ML-based systems that are Lighter, Clearer, and Safer, inherently aligned with a broad spectrum of compliance requirements. ACHILLES aims to create an efficient, compliant, and ethical AI ecosystem, addressing challenges related to privacy, security, fairness, and transparency.</p><p><br></p><p>The project proposes an iterative development cycle inspired by clinical trials, consisting of four modules focused on human-centric, data-centric, model-centric, and deployment-centric strategies. This approach seeks to enhance the performance and reliability of AI systems while ensuring compliance with legal and ethical standards. A key innovation is the development of a machine learning-driven Integrated Development Environment (IDE), which will streamline the integration between modules and promote the creation of responsible AI solutions.</p><p><br></p><p>Involving 16 partners from 10 countries, ACHILLES seeks to strengthen the European AI ecosystem, validating its applications in real use cases such as healthcare, identity verification, content creation, and pharmaceuticals.</p>

2024-06-06 – 2026-10-05

UniqueMark Operation

UniqueMark Operation

<p>The project is an enhancement of algorithms, designs, and enablement of rapid verification in databases, of the UniqueMark project, developed between INCM and ISR-UC, which delivered a proof-of-concept for unique assay markings on precious metals via punch/laser processes. The UniqueMark project aims to improve the safety of INCM&#8217;s contrasting marks in precious metal artefacts (the precious metals are officially designated: gold, silver, platinum and palladium), making them unique and unable to be copied and faked. One of the goals of the project is to allow the authenticity testing of the object to be done by goldsmiths, merchants, and consumers, armed with easy-to-use instruments such as smartphones.</p><p>Following the two previous projects, the UniqueMark Operation project emerges. The main objective of this project is the operationalization of the UniqueMark technology, with a view to its commercial launch by INCM. Thus, within the scope of this project, the main tasks of the project involve (1) carrying out exhaustive tests on a large number of precious metal artifacts, supplied by INCM, either using diamond particle punching or using laser equipment; (2) improvement of previously developed algorithms for identification and characterization of the unique distribution of particles (punching) or for the characterization of marking by identification of unique characteristic points created by the laser beam (laser marking); (3) improvement of the validation algorithms and (4) the creation of a database of images of assay markings. The database is expected to contain thousands of images, including high-resolution images acquired with dedicated equipment and images acquired with mobile phones. This database will be at the service of training and testing algorithms for detail extraction and marking recognition.</p><p>In short, the UniqueMark Operation project is a definitive step forward towards making UniqueMark technology a reality at the service of the consumer, creating a high security and anti-fraud instrument, but also innovative digital mechanisms to improve the consumer experience.</p>

2024-04-15 – 2026-04-15

CodeFace 2

CodeFace 2

<p>Project in development following a successful TrustFace proof of concept of CodeFace&#8211; technology that enables embedding a secret message within facial images. Building on the original CodeFace architecture, which utilizes deep neural networks to imperceptibly embed and reliably decode hidden messages in facial photos, the CodeFace 2 aim is to advance this technology to the next stage of development, focusing on enhancing security of portrait photographs on ID/travel documents, allowing integrity on-the-spot verification via smartphones&#8211; with or without internet access.&#160;</p><p>The CodeFace technology, developed at ISR-UC within the scope of projects in partnership with INCM, has created a system for encoding and decoding an image for covered communication, that is, with the technology in question it is possible to hide a secret message in an image and transmit it, so that the receiver can, using a camera, for example from a mobile device, decode the hidden information.&#160;</p><p>This technology was designed and optimized for the creation of a security element in ID and travel documents, for example passports or civil identification cards, such as the Citizen Card. This security element then makes it possible to encode a message hidden in the portrait photograph, with an almost imperceptible change in appearance, and that allows the inspection authorities, or any other authorised person, to validate the integrity of the photograph of the document, by being able to confirm that the message hidden in it corresponds to the expected message, by respecting a set of rules previously defined by the authority issuing the document.</p>

2024-01-01 – 2026-12-31

REMAIN

REMAnufacturing of deformable INdustrial products

The REMAIN Project, part of the Interreg Sudoe programme and co-financed by the Euro pean Regional Development Fund (ERDF), addresses the technological gap in traditional manufacturing sectors—particularly footwear, textiles, and furniture—by introducing advanced, cost-effective remanufacturing solutions. These sectors, heavily composed of SMEs, often struggle to adopt new technologies due to high implementation costs. REMAIN tackles this by developing a robotic system capable of disassembling and preparing used products for reintegration into production lines. The project also focuses on designing products with easier repair and recyclability in mind. As part of this, a comprehensive eco-design guide is being developed to help companies improve circularity and reduce waste. This guide will include practical steps for integrating remanufacturing into business models, thus contributing to sustainability goals and regulatory compliance within the EU Green Deal framework. Key innovations include a damage detection system using RGB vision and tactile sensors, as well as HW/SW platforms for robotic perception and automation. These will be tested in pilot actions, primarily within the footwear industry. A dedicated Digital Toolkit will support decision-making and strategic planning for companies looking to adopt these practices. AIDA CCI also supports knowledge dissemination through events like the industry seminar held in Aveiro, where project partners shared methodologies and preliminary market research insights. These initiatives help raise awareness about the environmental and economic benefits of remanufacturing.

2022-12-23 – 2025-12-31

Sustainable Stone by Portugal

Valorização da Pedra Natural para um futuro digital, sustentável e qualificado

Objectives: to enhance the relevant mobilizing and aggregating work that has been carried out in the context of the natural stone sector, for the creation of a new generation of products and production processes, strongly disruptive and innovative.

2021-10-01 – 2023-09-30

VISUAL-ID

Identidades Visuais Únicas em Gráficos, Imagens e Faces

The Visual-ID project emerge in the context of the partnership between the Imprensa Nacional-Casa da Moeda (Portuguese Mint and Official Printing Office), the University of Coimbra and Institute for Systems and Robotics (ISR) and includes two distinct, however complementary, areas such as “identification” and “authentication” using Computer Vision in commonly available devices equipped with cameras (e.g. smartphones and tablets), continuing the partnership already started at 2016, which has successfully accomplished several projects achieving outstanding outcomes. With a two-folded goal of offering a robust technology for single product identification and to fight against counterfeit, the project aims at creating visual identities with graphical codes and unique physical elements, able of being detected and validated through mobile devices, as the ubiquitous smartphones. It is intended to create stamps, labels or documents – either printed or digital -, with safer codes and taking advantage of the sophisticated potentialities of the UniQode® technology (icons, glitters, holograms, complex drawings). On the other hand, to face the need for trustworthy authentication of civil identification documents, it is expected to enhance the techniques developed in the TrustFace® project, namely for the generation of facial images with encoded data not visible for the naked eye and likely to be decoded and validated by smartphone-like devices. It is intended to move forward from the proof of concept, in which the possibility of encoding a huge amount of data in images compatible with ICAO standard was demonstrated, towards a robust and efficient algorithm that can be industrialized and that explores the latest aspects of the TrustFace® e CodeFace® technologies. The VisTeam group of the Laboratory for Computer Vision of ISR takes advantage of the artificial intelligence and machine learning techniques applied to Computer Vision, also resorting to research on the usage of different materials, that can enhance the physical-digital connection, namely in security materials such as holograms and inks containing glitter particles.

2018-01-01 – 2022-12-31

COMMANDIA

Collaborative Robotic Mobile Manipulation of Deformable Objects in Industrial Applications

In the SUDOE space there are many strategic industries that deal with deformable products, such as food, clothing, toys, or leather goods. The automation of the processing of deformable products is very difficult and expensive, and therefore many tasks are performed manually by human operators. Performing these tasks by means of robots would improve competitiveness and working conditions. However, current industrial robotic solutions are not sufficiently adequate to handle this type of products. This project proposes to cover this lack of innovation in the manipulation of deformable objects by defining, designing and implementing functionalities that allow current robots to manipulate these objects. The cooperation of the partners will result in the development of a multi-robot system that will be transferred, as a prototype to demonstrate application in two paradigmatic industrial sectors in the SUDOE space: meat processing and shoe manufacturing.

2016-10-01 – 2018-09-30

KhronoSim

System for Simulation and Test of Complex Systems

Concepts such as “Fourth Industrial Revolution (Industry 4.0)” and “Internet of the Things (IoT)” boasted into the technology speech like a blizzard, touching those who use and interest themselves of technology almost as much as those developing it. Such concepts are not surprisingly more used than understood; more are those using the concepts than those actually understanding their implications. Surprisingly enough, the increase of use of technology by the population at large, makes that security is not the least well-known aspect, though still not fully grasped however. Less well-known are the implications of complex systems working tightly coupled, with little or no human intervention, or possibility of human intervention, whatsoever. In such a scenario, testing components individually, one-by-one, is not sufficient to assert the correct functioning of the overall system. KhronoSim aims at developing a platform for testing cyber-physical systems in closed-loop. A platform that is modular, extensible and usable in multiple application domains. A platform featuring hard-real-time control, enabling the integration of simulation models to build a closed loop test environment and allowing the use of physical and virtual systems alike. Case study: simulation, control, and test of a sun-synchronous satellite.

2020-04-17 – 2022-04-17

HAIRBOT

Robotic System for Hair Transplantation

As part of its process of modernization and international expansion of its Viable Health has conceived and is implementing an ambitious project to development of a medical robot for hair transplant using combination of computer vision, mechanical and automation solutions. This project aims to promote the investigation of a robotic solution for the capillary extraction, unique and innovative, which materializes the strategy and vision of Saúde Feasible for the hair transplant sector using the FUE technique. the reality of hair transplant industry based on the FUE technique is faced with serious limitations of different natures, and to respond to these limitations and challenges, the HAIRBOT project aims to establish a set of technologies that make the concept of RFUE (Robotic Follicular Unit Extraction) technically feasible. The scientific intervention areas of this project include industrial robotics, vision artificial and computational learning, as it is expected that the project can Contribute to: 1. Explore the RFUE concept in the technological domain related to follicular extraction 2. Define and interpret human-machine interactions. 3. Disseminate fundamental research results at conferences and scientific journals, contributing to reinforce the quality of research Portuguese with the scientific community 4. Reinforce internal R&D skills and assert Viable Health as a cross-border innovation company

2020-01-21 – 2022-01-20

VIDROTEC 4.0

Indústria 4.0 na produção de vidro

The VIDROTEC 4.0 project arises with the aim of implementing the new emerging technologies of industry 4.0. This project thus aims to increase the efficiency of a hot and cold surface treatment system. Objectives: -Supervising the analysis of applications used in glass production. Study of image capture to be used in sprayers. -Support in methodologies and procedures to be implemented in the pilot unit. Study of different capture methods for computer vision to be used in the pilot unit. -Conception of the computer vision communication diagram and development of the thermal gradient capture system. --Development of the sensor system and digital twin communication. Support for the conception of computer vision. -Guidance and support in evaluating the process of embedding the electronics in the component. -Consulting and analysis of concepts applied to the pilot unit in spraying.

2020-01-01 – 2022-12-31

AGVStone

Desenvolvimento de um AGV para fábricas de pedra natural

The main elements of innovation proposed in this project are: 1-Development of a solution capable of transporting very large loads and transferring them to other platforms (transfer/handling mechanisms); 2-Development of a navigation and localization system capable of operating in harsh environments, and in particular, environments heavily polluted with dust and dirt. The most used sensory modalities involve the propagation of light (LIDARES and cameras) so robust solutions suitable for these environments must be used; -Development of localization and navigation models with the ability to learn and adapt to the variability of environments found in stone processing industries;

2019-09-01 – 2022-08-31

Zero.Mould

Integration of no manual strategies and technologies for high value automatization of mould precision adjustment

The objective of ZERO.MOULD is the development of systems and processes for with a view to reducing the defects / deviations accumulated in the different stages of production of an injection mold, raising the stage of mold production to a digital industry scenario, equipping the industry with prototype equipment with movement and control that allows assembly, polishing and computer vision able to mimic the operations carried out by the technicians, in order to revolutionize adjustment times and accuracy and also to captivate new engineers for this task, in order to leverage and enhance the industry of molds.

2018-09-01 – 2022-12-31

Morphing.Tech

Direct digital Manufacturing of automatic programmable and continuously adaptable patterned surfaces with a discrete and standardized composition

The company OPTIMA has over 16 years of activity, is dedicated to the design, development, production and commercialization of CNC equipment, for cutting, milling, machining and engraving. OPTIMA, within the scope of the technological surveillance activity it carries out, to the markets in which it operates, identified the growing need / demand for manufacturing processes that allow the production of multimaterial thermoplastic panels with variable thickness. The consortium led by the company OPTIMA, composed by the companies Periplast specialized in plastics extrusion, Luz Costa & Rodrigues manufacturer of extruded components and supplier of raw materials, GenialImpulso as? End-user? manufacturer of large components for the automotive sector, by ISR-University of Coimbra, University of Minho and CDRsp-IPLeiria as R&D entities, through the MORPHING.TECH project intends to develop a new article / manufacturing process for multi-material polymeric panels with thickness variable, through extrusion processes with configurable die and thermoforming on adaptive surfaces. The MORPHING.TECH project aims to produce multilayer polymeric panels of variable thickness, with different polymers (engineering materials, recycled and reinforced with natural fibers), for applications in the medical field, furniture, appliances, sports, technical parts, automotive and aeronautics . With the MORPHING.TECH project, the leading promoter, OPTIMA, positions itself as a supplier of innovative products in the field of polymer processing technology. Objectives: To develop and implement a manufacturing process in which the surfaces that originate molding zones are continuously adaptable and therefore do not depend on long machining processes. The concept is based on the Z-fitting of a stacked array of rods. This adjustment can be performed using servo motorized or hydraulic linear actuators. The dimension in Z of the elementary rods will be based on the 3D CAD design and will be verified in real time through computer vision systems. This process of obtaining molds can be applied to several molding processes, among other applications that are listed in a general way below, and in this project case studies in the field of thermoforming will be explored.

2018-01-01 – 2022-07-25

MUCAEN

Multi-Cam Capsule Endoscopy Imagery: 3D Capsule Location and Detection of Abnormalities

Wireless capsule endoscopy (WCE) is a medical technology, noninvasive, devised for the in vivo and painless inspection of the interior of the gastrointestinal (GI) tract. It is particularly important for the examination of the small intestine, since this organ is not easily reached by conventional endoscopic techniques. It consists of one or more miniaturized cameras, a light sources and a wireless circuit for the acquisition and transmission of signals. In a WCE exam, a patient ingests the capsule, and as it moves through the GI tract, propelled by peristalsis (a contraction of the small intestine muscles that pushes the intestine content to move forward), images are transmitted to a data recorder, worn on a belt outside the body. After about 8 hours, the WCE battery lifetime, the stored images, approximately 50.000 images of the inside of the GI wall, are transferred to a computer workstation for off-line viewing. We will focus on capsules equipped with multiple cameras. Multi-cam capsules allow improved estimation of the 3D localization due to the redundancy, and to the additional geometric and radiometric constraints. Multi-cam capsules allow the full estimation of 3D rotation and translation (including scale). Shape-from-shading and local models of the small bowel and colon will be exploited to constrain the estimates and reduce the uncertainty. Abnormalities will be detected based both on geometric and radiometric models and learning-based approaches.

2017-10-01 – 2022-03-31

ACHIEVE

AdvanCed Hardware/Software Components for Integrated/Embedded Vision SystEms

ACHIEVE-ETN aims at training a new generation of scientists through a research programme on highly integrated hardwaresoftware components for the implementation of ultra-efficient embedded vision systems as the basis for innovative distributed vision applications. They will develop core skills in multiple disciplines, from image sensor design to distributed vision algorithms, and at the same time they will share the multidisciplinary background that is necessary to understand complex problems in information-intensive vision-enabled applications. Concurrently, they will develop a set of transferable skills to promote their ability to cast their research results into new products and services, as well as to boost their career perspectives overall. Altogether, ACHIEVE-ETN will prepare highly skilled early-stage researchers able to create innovative solutions for emerging technology markets in Europe and worldwide but also to drive new businesses through engaging in related entrepreneurial activities. The consortium is composed of 6 academic and 1 industrial beneficiaries and 4 industrial partners. The training of the 9 ESR’s will be achieved by the proper combination of excellent research, secondments with industry, specific courses on core and transferable skills, and academic-industrial workshops and networking events, all in compliance with the call’s objectives of international, intersectoral and interdisciplinary mobility.

2017-01-16 – 2022-01-16

PRODUTECH R3

Task 6.1

PRODUTECH – Production Technologies Cluster - is an initiative promoted by the Portuguese Industry of Manufacturing Technologies. This Cluster comprises companies that are capable of addressing competitiveness and sustainability challenges by delivering innovative, flexible, integrated and competitive solutions in response to the manufacturing industry’s needs. PRODUTECH embodies collective efficiency strategies aimed at innovating, qualifying and modernizing the developers and users of manufacturing technologies. As a key partner in several projects, PRODUTECH promotes initiatives and actions fostering the international competitiveness of the Portuguese economy, and boosts the cooperation between companies and other relevant stakeholders in a structured way. Main objectives: - Generation of knowledge in various enabling technologies so that industrial companies can invest in the development of online monitoring systems for advanced manufacturing techniques with feedback for self-correction for the regulation / optimization of process variables and parameters; - Non-destructive inspection and testing systems (NDI / NDT) applied to highly complex defects and geometries. Use of enabling technologies such as artificial vision (exploring hyper and multi-spectral cameras), laser scanning, among other technologies. Development of advanced image processing solutions and algorithms; - Non-destructive inspection and testing systems (NDI / NDT) applied to high-rate processes. Analysis of the application to flexible cells that are automatically adaptable to variations in the product or its conditions in the cell (variations in position or dimensions, for example); - Communication of inspection and monitoring systems with other systems (Smart Sensors, IoT, etc.) foreseeing their integration into Industry 4.0 systems. Take into account the standards and protocols according to Industry 4.0 in developments.

2016-07-01 – 2019-06-30

VisArthro

Computer Vision For Computer Aided Arthroscopy

Arthroscopy is a Minimally Invasive Surgical (MIS) procedure for treatment of damaged joints in which instruments and endoscopic camera (the arthroscope) are inserted into the articular cavity through small incisions (the surgical ports). Arthroscopic procedures are very difficult to execute because of indirect visualization and limited manoeuvrability inside the joint, with novices having to undergo a long training period and experts often making mistakes of clinical consequences. The project aims to overcome this drawback by conducting research in computer vision techniques to build a system for Computer-Aided Arthroscopy (CAA) that will guide the surgeon throughout the procedure. Such system will exclusively rely in processing the arthroscopic video, being the first of the kind not requiring additional intra-operative sensing modalities such as opto/magnetic tracking. This is a crucial feature to assure broad adoption among surgeons, ultimately contributing to disseminate and improve the clinical outcome of arthroscopy, with indisputable benefits for the patient. Our research will be motivated by two specific procedures: the reconstruction of the Anterior Cruciate Ligament (ACL) in the knee and the removal of Femur Acetabular Impingements (FAI) in the hip. ACL tear is a common pathology for which arthroscopy is the standard treatment (> 300000 cases per year worldwide). The procedure consists in replacing the torn ACL by a substitution graft that is pulled into the joint through a tunnel opened with a drill (Fig. 1). Placing this tunnel in the correct anatomical position is crucial for the knee to fully recover its functionality [1,2]. Recent studies show that the dominant surgical technique, the transtibial approach, only reaches optimal tunnel placement in 61% of the cases [1]. FAI occurs when the ball shaped femoral head rubs abnormally in the acetabular socket, which in 91% of the cases is caused by an excess of bone tissue in the femur head-neck that creates a bump known as CAM impingement [3] (see Fig 2). The treatment is surgical and consists in removing the CAM to restore the ball shape to the femur-head. Unfortunately, such reshaping via arthroscopy is extremely difficult to accomplish being only performed by a small number of highly skilled surgeons (e.g. in Portugal there are only two surgeons doing the procedure).

2014-09-02 – 2015-12-31

Modeling and calibration of a stereo endoscope

Modeling and calibration of a stereo endoscope

2012-05-01 – 2015-04-30

PTDC/EIA-EIA/122454/2010

Calibration and 3D Reconstruction in Omnidirectional Catadioptric Systems

Vision, Imaging and 3D graphics systems can be classified into two types according to the geometry of image formation: central and non-central. Central systems, also known as single viewpoint systems (SVP) are systems where all the projecting rays intersect at a single point, the effective viewpoint, which is the case, for example, of the common perspective cameras. In this case the effective viewpoint is the center of projection of the camera. One particular class of non-central imaging systems are omnidirectional vision systems. These systems have the advantage of providing images with wide fields of view which are advantageous for many applications ranging from surveillance, computer graphics, rendering, robotics and also in some medical applications. One of the common approaches to obtain omnidirectional images is to use catadioptric imaging systems (CIS). Catadioptric imaging systems combine mirrors and lenses. However there are CIS that are central, as shown in [Bak99]. These are obtained only with very specific configurations namely: -a perspective camera whose center of projection is placed on the focus of hyperbolic or elliptical mirror; -- an orthographic camera aligned with the axis of a parabolic mirror. These configurations require the careful placement of the camera and/or a special camera (orthographic). All the other configurations using mirrors lead to non-central CIS which implies that the rays do not intersect at a single point. As mentioned non-central imaging systems occur not only in CIS that but also in many other cases and applications. Cameras using fish eye lenses are also non-central as are cameras operating immersed in a fluid or interfacing other refractive environments. That is the case of medical applications such as endoscopy, colonoscopy or minimally invasive surgery. It is also the case in underwater robotics. 3D reconstruction from non-central images has been studied namely in [Mic04], [Gon04], [Kan06] and [Din09]. In all these cases multiple non-central images acquired with multiple mirrors (or with a moving mirror) used. 3D reconstruction has also been performed using a single image and using distortion to estimate depth. This project aims at studying calibration and 3D reconstruction in the following non-central cases: --Using 3D cameras (for example time-of-flight cameras) looking at a curved mirror or fish-eye lens to perform 3D reconstruction; --Having multiple perspective cameras looking at the same curved mirror to perform 3D reconstruction; In the case of 3D cameras we will also study the sampling of the plenoptic function performed by these imaging systems namely in what concerns the radiometric issues. To the best of the PI knowledge there are no results in the literature modeling the combination of 3D cameras with curved mirrors or modeling the use of 3D cameras in other non-central configurations namely when immersed in non-homogeneous environments. Also 3D reconstruction in these cases has not yet been modeled. The case of 3D reconstruction using multiple perspective cameras looking at a single curved mirror has not been fully addressed in the literature. In this case there are issues related to radiometric aspects namely with the resolution and focus that have not been studied. The PI has, in the last few years, addressed several problems related to central and non-central catadioptric systems ([Gon04], [Gon07a], [Gon09], [Gon07b] ) including modeling, calibration and 3D reconstruction. The use of a single mirror or lens has the significant advantage for applications of allowing more compact imaging systems. On the other hand 3D data is useful in many applications. In medical applications, namely in endoscopy/colonoscopy 3D images would significantly facilitate not only the navigation of the tool but also the recognition of the pathologies. Colonoscopes/Endoscopes are non-central systems. Furthermore they operate usually immersed in fluid which further accentuates the non-centrality. Surveillance applications also benefit from 3D data since recognition of human behavior can be significantly improved. The results of the project will allow better modeling of image formation in non-central imaging systems which will also benefit 3D Computer Graphics. The topics of this project correspond, partially, to the problems addressed by the Ph.D. students Pedro Miraldo and Luís Perdigoto on their ongoing dissertations.

2010-04-01 – 2013-09-30

UniProjection

Unified Projection Model of Non SVP Systems - Application to Endoscopy and Graphics

The projection model (PM) of a vision system is a function that relates a 3D point and its projection in image. The PM is central or noncentral whenever all incident rays intersect each other in a unique point (central) or not (noncentral). This is of key importance for the type of methods one can use in the visual system, since most of them are only applicable with accuracy for central projection systems. Furthermore, the refraction of light in liquid mediums as well as the effect of scattering augments the non-centrality of the projection, possibly making the usual central assumption unrealistic. This project aims to achieve two main objectives. In the theoretical point of view, it aims to achieve the derivation of a unified and parametric PM for noncentral cameras using mirrors, lenses or passing through liquid environments. The existing PMs are usually ray-based (non-parametric) and as a consequence difficult and heavy to calibrate. The PM must be also applicable to illumination path calculations since the basic physical phenomena are the same as for image formation. Several researchers that worked at the project UniProjection (FCT funded) are now placed at the industry in regional enterprises (e.g. Project Box -Viseu) and national enterprises (e.g. Vison Box - Lisboa) due to their high level of expertise in Vision acquired during their research phase at the ISR project. It was possible to fix several specialized jobs nationally and regionally.

2010-03-25 – 2012-09-28

PTDC/EIA-CCO/108791/2008

Identity Recognition using 4D-Facial Dynamics

The human face has been the subject of tremendous scrutiny in the fields of human cognition, computer vision, image processing and computer graphics, and is perhaps the most extensively researched facet of our body. This is hardly surprising given the importance of its function in social interactions. The face is the primary medium for conveying identity, cognitive state, emotive intent and disseminating affective responses. Although humans use language as the main channel for conveying facts and schemes, it does not match the face’s capacity for communicating emotions: - “The face is rich in communicative potential. It is the primary site for communicating emotional states; it reflects interpersonal attitudes...” - The study of how facial expressions transmit affective information was pioneered in behavioural psychology by Ekman and his associates. From these studies it was discovered that facial expressions are uniquely human, basic emotional expressions are consistent in exhibition and interpretation across all demographics, and emotional messages are constructed by the actions of certain facial muscles. - Due to its complex dynamic nature, the study of the face from a computational perspective has resulted in increasingly more sophisticated tools in non-rigid object modelling and tracking, object parameterisation and recognition, occlusion handling and extracting invariant features in real-world settings. Automated facial analysis has been motivated by some very desirable applications, which consider both its static nature, such as identity recognition, and its dynamic nature, such as expression recognition, visual speech recognition and the realistic animation of virtual humans. This analysis has also paved new insights and tools in medical applications, behavioural science, security, education and human computer interaction. - Much of this analysis has been conducted on static2D or 3Dimages or short2D image sequences. However there has been very little work in investigating facial dynamics in video-rate 3D data. The advantages of 3D over 2D data in pattern recognition tasks have been largely considered as a means to overcome variations in pose and illumination. However, 3D information over time also provides us with a complete description of how an object deforms in 4D spatiotemporal space without the loss of information which is incurred as part of the 2D image projection process. With respect to the human face, one important use for this is to analyse the ways in which individuals can, or are able to deform their face while performing expression or speech. We can use this analysis to explore the similarities and idiosyncrasies of facial motion across individuals, which has important applications in physiological and clinical studies. We can also use this to generate individualized dynamic facial models for highly realistic animation. Another direction of research stems from the question of whether it is possible to characterise an individual based on their facial motion. For example, how can we quantify the similarly between two peoples’ smiles? Which expressions best discriminate individuals? Is it possible to build a prototypical 4D model of how people smile? How are individual differences reflected as a deviation from this model? To the best of our knowledge, these questions have received a very small amount of attention in 2D image analysis, and none in 3D sequences. From a biometrics point of view, the concept of recognising a person based on facial motion is attractive; since facial movements comprise a complex sequence of muscle activations, it is almost impossible to imitate another person’s facial expressions and these facial motion characteristics are unique to an individual. Furthermore, the use of facial motion is in certain aspects more robust to fraudulent attacks than current static face recognition algorithms, which fail if presented with a physical model of a person’s face. Facial dynamics are also independent of lighting, pose and appearance changes (such as wearing make-up), which are apparent in real-world environments. In experimental psychology, determining the precise role of facial motion in determining identity is still largely unknown, and is actively pursued. By exploring this notion using computer vision techniques, we will be able to evaluate the strength of the dynamic cue in identity recognition.

2010-02-15 – 2013-08-14

MORFEU

Multi-Objective Robot Fleet for improved commUnication

In many disaster situations the human intervention may involve several risks. These threats may derive from different factors such as damaged infrastructures which can potentially collapse at any instant, inflicting damage to working rescue teams. In addition to this, a hazardous environment may also result from broken gas pipes, biohazard materials and other harmful substances.The usage of robotic rescue agents is a powerful tool for rescue operations where no human casualties need to be at risk. These agents can perform common rescue operation tasks and generally operate with reconnaissance purposes, gathering information from many possible sensors. Similarly to a human rescue team, a team of robot agents moves through the disaster impact zone in search of relevant information which may include the search of trapped victims. An important requirement for any rescue team is that some communication infrastructure is always, or at least frequently available. This requirement is fundamental for non-autonomous teams and, particularly, for timely transmission of critical data. The proposed rescue scheme considers multiple robot teams equipped with traditional sensors such as a laser range finder, an omnidirectional video camera and an omnidirectional wireless antenna using the IEEE 802.11 standard. Each team should be remotely controlled, as whole, by a human operator, thus reinforcing the importance of connectivity maintenance. The connectivity requirement should autonomously be met by each individual robot which progressively adjusts its own position in order to optimize the network connectivity. As an additional objective, each robot should also maintain a relative positioning system. The robot position can be obtained through triangulation amongst other team nodes, such that the entire team is aware of all its robots’ positions. Regarding the desired positioning system, one important aspect is that certain nodes, anchor nodes, possess additional positioning information which can be obtained from sensors such as GPS, when available, or even from known reference points when they are identified. As previously explained, there is a hybrid navigation mode for robots, where an entire team is remotely controlled and where within each team there is an autonomous node movement. This movement is based on each robot’s coordinates with connectivity and relative positioning maintenance objectives as a concern. This aspect motivates the need of a communication method that efficiently achieves suitable performance for the correct operation of the mobile nodes. For this purpose, Wireless Mobile Ad-hoc networks are going to be used. As previously explained, there is a hybrid navigation mode for robots, where an entire team is remotely controlled and where within each team there is an autonomous node movement. This movement is based on each robot’s coordinates with connectivity and relative positioning maintenance objectives as a concern. This aspect motivates the need of a communication method that efficiently achieves suitable performance for the correct operation of the mobile nodes. For this purpose, Wireless Mobile Ad-hoc networks are going to be used. Mobile Ad-hoc networks (MANETs) aim at providing a self-configured, self-maintained and self-administrated reliable wireless network without the need of any existent infrastructure. In addition to the support of the already mentioned remote controlled of teams, the final purpose of such network may be different depending on several distinct node objectives. In a rescue situation, a MANET may be used to provide connectivity amongst every agent related with the operation for simple control messages exchange and for supporting all the possible services that these agents may require in order accomplish their mission such as video and audio transmission.Routing protocols represent an important role in how MANETs behave and have a major impact in network performance. Hence, the decision of an appropriate routing protocol is one challenge from which the entire rescue operation may have to rely on. Regarding the proposed rescue approach, the wireless network benefits from the organized robot teams which by themselves keep as an objective maintaining network connectivity. However, from the network perspective, connectivity may not be sufficient to efficiently support the rescue services used during the operations. One important challenge is to analyze the network performance such that, robot nodes are capable cooperate with the routing protocol, helping to achieve the required performance for all the necessary operations by changing their current position according to the existing information. Despite the fact each robot team aims at keeping connectivity between all its nodes, several rescue teams, simultaneously, may be required to target a specific area.

2006-01-02 – 2008-12-31

EU STREP

Perception on Purpose

The ease with which we make sense of our environment belies the complex processing required to convert sensory signals into meaningful cognitive descriptions. Computational approaches have so far made little impact on this fundamental problem. Visual and auditory processes have typically been studied independently, yet it is clear that the two senses provide complementary information which can help a system to respond robustly in challenging conditions. In addition, most algorithmic approaches adopt the perspective of a static observer or listener, ignoring all the benefits of interaction with the environment. This project proposed the development of a fundamentally new approach, perception on purpose. which is based on 5 principles : - visual and auditory information should be integrated in both space and time. -Active exploration of the environment is required to improve the audiovisual signal-to-noise ratio. -The enormous potential sensory requirements of the entire input array should be rendered manageable by multimodal models of attentional processes. -Bottom-up perception should be stabilized by top-down cognitive function and lead to purposeful action. -All parts of the system should be underpinned by rigorous mathematical theory, from physical models of low-level binocular and binaural sensory processing to trainable probabilistic models of audiovisual scenes.

2018-09-24 – 2020-08-24

TrustFaces

TrustFaces

The TrustFaces project derives from the TrustStamp project, completed in June 2018, in partnership with the INCM - Portuguese Mint and Official Printing Office. This project aims at a partnership for research and development in the area of reliable labels for the certification of products and identification (ID) documents, namely in the authenticity certification area, more specifically in the creation of stamps and encoded images of faces of people, with applications in personal ID documents. In the current state of the TrustStamp project, the methods for creating images of trust stamps that can encode hidden information through the association of a dictionary with dithering patterns have been developed, resulting in the creation of UniQode® codes. The method is very flexible and allows additional layers of coding, encryption, error correction and redundancy to be added on a simple hierarchical structure in order to design the security of messages according to the application for which it is intended. It is therefore possible to create stamps with different levels of security, suitable for products and solutions with specific needs. In addition, this technology allows to associate several media and types of prints, namely, associating a standard printing on paper with a standard printing in hologram, increasing safety drastically, by the added difficulty that is to produce a system to breaking these stamps or, through printing with inks enriched with metal particles (glitter) for the generation of unique and irreproducible random patterns (resistant to counterfeiting and fraud).

2019-06-04 – 2020-06-04

UNIQODE

UNIQODE

Este projeto é a continuação do projeto TrustStamp, alargando o seu âmbito e permitindo dar resposta à crescente e iminente necessidade de acelerar a adaptação industrial da INCM para a introdução nos seus atuais e novos produtos da tecnologia UniQode. Por um lado, pretendem-se explorar mais potencialidades do UniQode, nomeadamente melhorando o processo de criação de identidades visuais cada vez mais complexas, quer disfarçando o UniQode em identidades visuais de grandes dimensões, em suporte papel ou holográfico, quer aumentando drasticamente as dimensões do próprio código gráfico com a crescente complexidade na sua reconstrução e, por outro lado, pretende-se com este projeto a construção de uma linha de produção industrial mais rápida, mais eficiente e mais rentável que permita reduzir drasticamente o tempo de cada projeto individual, padronizando a sua definição e configuração, através do desenvolvimento de APIs para integração com o sistema de design e produção de selos e rótulos. Estes dois principais objetivos, isto é, por um lado, alargar as possibilidades e, por outro lado, restringi-las pela via da padronização, só são possíveis pelo facto de permanecerem ainda complexos desafios na exploração das potencialidades do UniQode, requerendo um esforço de investigação e inovação não negligenciáveis, ao mesmo tempo que a padronização trará ao projeto o foco necessário ao ambiente de produção industrial. Os principais objetivos deste projeto são, pois, elencados seguidamente: investigar metodologias de desenho e configuração de uma mensagem visual bidimensional baseada em icons. Esta mensagem visual é composta por: um mapa de localizações para o posicionamento de icons; definição de um alfabeto de símbolos visuais (icons) que podem ser agrupados para a definição de palavras de um léxico visual. investigar metodologias e algoritmos de aprendizagem máquina (machine learning), nomeadamente usando redes neuronais convolucionais (CNN – convolutional neural networks) e algoritmo do vizinho mais próximo (kNN – k-nearest neighborhood), para a deteção e identificação dos icons, permitindo estudar o limiar máximo de semelhança e sobreposição entre símbolos diferentes, para melhoria do processo de descodificação dos icons. investigar metodologias e algoritmos para a deteção de elementos visuais de pequenas dimensões em hologramas, nomeadamente aplicando técnicas de machine learning e/ou inteligência artificial investigar metodologias e algoritmos para a deteção de códigos gráficos impressos em ambientes visuais complexos (por um lado, incluindo ambientes visuais onde a proporção do graphic code é reduzida ou muito reduzida quando comparada com a dimensão do desenho geral e, por outro lado, incluindo identidades visuais onde o graphic code está embutido em identidades visuais de elevada frequência espacial e recorrendo a paletas de cor fortes) investigar metodologias e algoritmos para a reconstrução de códigos gráficos, baseados em pixéis, de grandes dimensões espaciais investigar metodologias e algoritmos para a criação de identidades visuais que sejam simultaneamente um UniQode baseados em pixéis e um QRcode, respeitando, assim, a norma desenvolver software industrial, nomeadamente APIs com algoritmos capazes de codificar e descodificar códigos UniQode, quaisquer que sejam as especificações técnicas envolvendo códigos gráficos baseados em pixéis e/ou icons, holograma e padrão de tinta com glitter (tendo em consideração o conjunto de variáveis, definições e especificações consideradas para uma identidade visual genérica). desenvolver uma API industrial para a definição e configuração das regras da camada de mensagem, permitindo um nível de complexidade configurável. desenvolver uma API para a definição e configuração de uma identidade visual genérica, integrando os módulos e camadas anteriormente desenvolvidas.

2014-01-01 – 2014-12-31

UrbanScan

3D modeling of urban scenes with a new stereo approach

In this project we propose a mobile reconstruction system based on 3D vision algorithms for obtaining models of entire city streets. The novelty with respect to previous systems is the development of a processing pipeline able to conciliate reconstruction accuracy with fast execution &ldquo;on-the-fly&ldquo;. If succeeded, the project will provide the grounds of a scalable solution for the 3D modeling of entire cities, which has a broad range of applications, as proven by the interest of major players like Google. In addition, the methods and techniques herein developed will be highly relevant for autonomous robots and intelligent transportation systems, that can highly benefit from real-time 3D perception of urban environments.

2009-12-31 – 2011-12-30

Line RDFixer

Ver + Saúde: Line RDFixer

RDfixer is a software solution for correcting the distortion of endoscopic images in real-time. It can be deployed in a plug-and-play processing device connected between the camera control unit (CCU) and the display or, in alternative, it can be integrated by the manufacturer of the endoscopy equipment to run in the CCU. RDFixer brings direct benefits to the practitioner/surgery by: (i) providing a better perception of relative depth and distances. (ii) Presenting the correct shape of the anatomic structures. (iii) Ultimately reducing the learning curve for new practitioners.

2008-12-30 – 2011-12-30

ArthroNav

Computer Assisted Navigation in Orthopedic Surgery using Endoscopic Images

The 3D reconstruction of scenes using monocular video sequences is a classical problem in computer vision. The literature covers, not only 3D reconstruction, but also related topics as camera calibration and 3D registration. However most of the described methodologies were developed for perspective images of everyday scenes. The goal of this project is to extend this framework to endoscopic video sequences in the context of computer aided orthopedic surgery (CAOS). Our target application is the visualization and navigation during the reconstruction of the Anterior Cruciate Ligament (ACL) using arthroscopy. Since we aim to develop methods to extract 3D information from endoscopic images, the project outcome will be useful for many other minimally invasive medical procedures.

2013-04-09 – 2015-04-09

FRIeND

Facial expRession analysis for INteraction and Demency detection

The scientific aim of this collaboration is to explore the use of facial expression analysis in two stages: • improve human-robot interaction mechanisms with an emotion recognition system, • explore the use of facial expression recognition in the detection of early stages of Alzheimer disease

2019-07-01 – 2019-11-30

Prozis

Prozis Challenge Project

The primary goal of this R&D project, a partnership between PROZIS.TECH and the Institute of Systems and Robotics, is the development of a "Fitness Station" completely disruptive in view of the state of the art, capable of capturing an athlete's movements seamlessly through computer vision and AI / ML. The project aims the creation of a database of images (2D / 3D) and / or three-dimensional postures acquired by the technology that is deemed most effective, the develop a computational solution, non-intrusive and fully supported in RGB images, oriented to the reconstruction and modeling of the body “pose” of athletes in a semi-environment controlled and also the develop technologies and machine learning algorithms for the study of kinetics and kinematics of the movement of athletes, namely in performing the “Squats” exercises, "Burpees", "Push Ups", "Sit Ups" and "Jumping Jack".

2019-01-01 – 2027-05-31

Computer Vision Systems

Computer Vision Systems

The primary goal of this R&D project, a partnership between AtoBe – Mobility Technology S.A. and the Institute of Systems and Robotics, is the development of vision-based computational solutions with direct application in traffic analysis and monitoring in highway scenarios and also in Urban scenarios. The project aims the development of deep-learning based solutions for abnormal event detections in highways, detection, tracking and semantic classification of vehicle’s class in surveillance cameras and free-flow tolling systems and vehicle occupancy detection for HOV/HOT lanes enforcement and tolling.

2018-01-01 – 2018-12-31

Computational Vision Systems for ITS

Computational Vision Systems for ITS

The primary goal of this R&D project, a partnership between Brisa Inovação e Tecnologia, S.A and the Institute of Systems and Robotics, is the development of vision-based computational solutions with direct application in traffic analysis and monitoring in highway scenarios and also in Urban scenarios. The project aims the development of robust solutions for abnormal event detections in highways, semantic classification of vehicle’s class in surveillance cameras and free-flow tolling systems and on the development of HOT/HOV tolling and enforcement systems. In urban scenarios, the project aims the development of vision-based solutions for parking lot and urban street vehicle parking management and the improving of urban city traffic through efficient crossroad semaphore control using vision.

2011-09-01 – 2014-12-31

Long Deck

Long Deck Suspension Bridge Monitoring - A Vision-based System for the 25 Abril Bridge

GOALS: Implementation of a Smart Structure (Eurocores programme &ndash; Smart Structural Systems Technologies (S3T), ESF). Development of a non-contact measurement system to monitor, in real-time, the full-motion of large civil structures (e.g. long deck suspension bridge). OUTCOMES: PhD Concluded, 2012. Monitoring solution under evaluation on site. Strengthening collaboration with LNEC

2011-09-01 – 2012-02-29

Facet

Facet Biometric Face Framework

MOTIVATION: Promotion and enhancement of skills to support the Entrepreneurship and Innovation. Exploitation of commercial aspects of research projects developed in R & D units FACET Purpose: Development of a SDK oriented to facial biometric applications.

2010-01-01 – 2011-01-01

ITraffic

Automatic Traffic Event Detection for the BRISA Telematic System

In order to support safe and efficient driving, it is important to classify the behaviours of vehicles and to understand their interactions on typical traffic scenarios. Until recent years, this task was performed by human operators at traffic control centers, but the huge increase on the number of available cameras requires automatic traffic surveillance systems In the last decades, one of the most important efforts in ITS research has been the development of visual surveillance systems that could help reduce the number of traffic incidents and traffic jams in urban and highway scenarios. Although the large number of systems based on different types of sensors and their relative performance, vision based systems are very useful to collect very rich information about road traffic. Brisa has recently built a Operational Coordinate Center (CCO) and is in the process of installing more than four hundred traffic surveillance cameras as part of his Traffic Telematic System (STR). The primary goal of the project was the detection and tracking of potentially anomalous traffic events along the highway roads. By anomalous events we mean the detection of vehicles that stopped on the highway, vehicles driving in lane's opposite direction and also vehicle that are constantly switching between lanes. The system should be able to identify each vehicle and track its behaviour, and to recognize dangerous situations or events that might results from a chain of such behaviours. The system should be robust to illumination changes and small camera movements, being able to robustly track vehicles against occlusions and crowded events. The promising results obtained with the project BRISA I gave us the opportunity to embrace a solid partnership for R&D through the creation of a ISR-BRISA R&D Laboratory totally funded by BRISA on a annual contract basis, renewable, with the purpose of developing solutions for a wide range of research challenges that needs to be addressed on ITS.

2009-08-25 – 2011-08-25

GenePreDit

Automation Platform for Collection and Preparation of Biological Samples

This project is mainly a technological project on which the ISR is involved only at a scientific&technological supervision level. The project was submitted by Gene Predit S.A. to QREN 2009 under the scientific supervision of ISR team members. The purpose of the project is the development of a robotic system with multiple degrees of freedom for the automatic collection and preparation of biological samples (drosophilae flies) to be used on the identification of genetic biomarkers associated to certain pathologies and the identification of drugs for the treatment of certain diseases.

2005-06-25 – 2008-06-25

AI_FI

Advanced Interaction Using Facial Information

The main aim is to investigate whether and how human facial gestures (e.g., blink, smile, etc.) and gaze could be included into standard HCI systems as new modes of HCI and as providers of context-discriminative information for revealing how the user feels (e.g., pleased, tired, etc.). The intended approach is to design a novel facial-information analyser and to test its usefulness in different scenarios. The core technical aim is to develop this novel facial-information analyser which would process human-face image sequences to detect the user’s facial gestures, their temporal patterns and the user’s gaze direction, and then to fuse and interpret them in terms of command/affect/mood descriptive interpretation labels in a user-profiled and user-point-of-regard-sensitive manner.

2005-05-01 – 2006-11-01

BRISA

Automatic Detection of Incidents in the Brisa Telematic System

Brisa has recently installed an Operational Coordinate Centre (OCC) and is in the process of installing more than four hundred traffic surveillance cameras as part of his Traffic Telematic System. The objectives of the project was the development of computer vision and image processing algorithmic solutions to be used on the smart surveillance cameras installed by Brisa along the highways. The primary goal of the project is to detect and track potentially anomalous traffic events along the Brisa&rsquo;s highway roads. By anomalous events we mean the detection of vehicles that stopped on the highway, vehicles driving in lane's opposite direction and also vehicle that are constantly switching between lanes. The system should be able to identify each vehicle and track its behaviour, and to recognize dangerous situations or events that might results from a chain of such behaviours. The system must be robust to illumination changes and small camera movements, being able to robustly track vehicles against occlusions and crowded events.

2016-11-01 – 2018-06-01

UniqueMark

UniqueMark

This project aims to improve the safety of INCM's contrasting marks in precious metal artefacts (the precious metals are officially designated: gold, silver, platinum and palladium), making them unique and unable to be copied and faked. One of the objectives of this project is to allow the test of authenticity of the object to be made by goldsmiths, merchants and consumers equipped with verification instruments with a high level of ubiquity and therefore relatively easy to handle, inexpensive and simple to acquire, namely basic lenses or smartphone cameras. This is a research and development project funded by the Imprensa Nacional Casa da Moeda (Portuguese Mint and Official Printing Office), the official portuguese state company in charge of providing fundamental goods and services to the Portuguese State, such as minting metallic coins and publishing the Official Journal. Security documents such as the Portuguese Citizen Card and the passport are also produced by INCM as well as the authentication of precious metals artefacts.

2016-11-01 – 2018-12-01

Card3DFace

Card3DFace

This project intends to create a 3D face printing system on cards. As for printing on polymer cards, the 3D effect is created by a lenticular structure made in the production phase of the card. The expected innovation for this project concerns the generation of the three-dimensional model of a person's face, which will be accomplished by manipulating photographs captured using light field cameras, also called plenoptic cameras, and choosing the number of views and their optimization, as well as their viewing angles, in order to create the best three-dimensional effect possible. This is a research and development project funded by the Imprensa Nacional Casa da Moeda (Portuguese Mint and Official Printing Office), the official portuguese state company in charge of providing fundamental goods and services to the Portuguese State, such as minting metallic coins and publishing the Official Journal. Security documents such as the Portuguese Citizen Card and the passport are also produced by INCM as well as the authentication of precious metals artefacts.

2016-11-01 – 2018-05-01

TrustStamp

TrustStamp

This project intends to develop verification tools to be applied on INCM trust stamps, to confirm authenticity and protect products against counterfeiting. The verification of the authenticity of the trust stamp will be done using a mobile application (smartphones and tablets), due to its high ubiquity and personal ease of use. The limits of verification using the camera of these devices, as well as the robustness of the application to demanding ambient conditions (illumination, stability and relative position of the camera relative to the seal) will be studied. It is also intended to explore coding solutions that allow to dispense the existence of databases with the unique standards by seal. This is a research and development project funded by the Imprensa Nacional Casa da Moeda (Portuguese Mint and Official Printing Office), the official portuguese state company in charge of providing fundamental goods and services to the Portuguese State, such as minting metallic coins and publishing the Official Journal. Security documents such as the Portuguese Citizen Card and the passport are also produced by INCM as well as the authentication of precious metals artefacts.

2024-01-01 – 2026-12-31

Winwork

Winwork

<p>WinWORK is a cross-border Interreg VI-A Spain&#8211;Portugal (POCTEP) 2021&#8211;2027 project that promotes health, quality of life and well-being in workplaces through a multidisciplinary, stakeholder co-creation approach. The consortium (led by Polit&#180;ecnico do Porto) designs and pilots interventions grounded in ergonomics, physical-activity promotion and psychosocial risk prevention, supported by immersive digital tools&#8212;notably VR-based interactive scenarios aimed at skills such as mindfulness/stress management and socio-emotional competencies. Pilot actions are planned across six organizations in Portugal and Spain, spanning industry, services and healthcare. University of Coimbra participates through interdisciplinary teams, including researchers affiliated with the Institute of Systems and Robotics (ISR-UC), contributing engineering and technology expertise&#8212;particularly in the development/adaptation of immersive learning tools and interactive applications that underpin the VR component of the intervention.<br></p>