Video-based Computer-Aided Arthroscopy and Orthopeadic Surgery:

The ISR-UC created the first effective concept for accomplishing navigated arthroscopy by combining real-time video processing for accurate 3D measurements on the anatomy, with augmented reality for overlaying meaningful guidance information in the intra-operative video.

The Video-based Computer-Aided Arthroscopy (VCAA) builds on advanced research in fundamental 3D Computer Vision that led to several articles in the most prestigious conferences and journals in the field as well as to 5 patents already granted in several geographies. These patents are licensed in exclusive to Perceive3D, S.A. (P3D) that is a spin-off company founded by two researchers of ISR-UC. P3D was considered a highly innovative company and awarded with the SME Instrument Phase 2 by the EU Comission, and received the Bartolome de Gusmão Award by Instituto Nacional de Propriedade Intelectual (INPI).

First ever method for digital printing of stretchable electronic materials:

• Large application potential:  Conductive inks for stretchable electronics, portable health monitoring,  e-textile, printed batteries, and printed antennas

• Joint Patent between UC(80%) and CMU(20%), pending in US, EU, and Korea 

 Liquid metal fusion with conductive inks and pastes,  WO2019055680, https://patentscope.wipo.int/search/en/detail.jsf?docId=WO2019055680)

• Publication in the highly prestigious Advanced Materials Journal with impact factor of 27.5

The above patent, and the formulations for the conductive inks, are the first methods that allow direct printing of stretchable electronics, and are expected to cause a positive impact in fabrication of thin film and flexible devices for biomonitoring, for flexible OLEDs, Energy harvesting films, and energy storage devices.  Disposable electronic tattoos are a major development in patient for safe and reliable patient monitoring. Therefore, we are expecting economic exploitation through licensing and commercialization of these inks through national and international companies.

A ongoing project is being carried out with Glint the major hospital equipment manufacturer in Portugal to transfer the technologies already developed and carry further advancements  in specific applications.

Novel Computing for Artificial Multimodal Perception Algorithms:

• Large application potential: The novel computing approach allows the implementation of energy efficient inference engines on embedded and low power hardware, as well as easing the load on data centres, exploring trade-offs between energy, time and precision. The slowing down of Moore’s Law and evermore demands for probabilistic computing sets the stage for a growing application potential of this novel approach beyond artificial perception in robotics.
• Publication in the prestigious International Journal of Approximate Reasoning with impact factor of 2.678, and a book part of Springer Tracts in Advanced Robotics book series.

Artificial perception plays a key role in the development of Intelligent Systems and Robotics. The sensation-cognition-action loop needs to deal with uncertainty, and probabilistic approaches provide a robust solution, so novel solutions in Bayesian Computation have been pursued. A key contribution in artificial multimodal perception for cognitive systems was on the actual way Bayesian computations are done. We are developing artificial perception algorithms, including integrated multisensory computational models. These include probabilistic approaches towards a new generation of algorithms to deal with uncertainty, ambiguities and conflicts inherent to the perceptual process that promote intelligent and adaptive decisions on actions in the physical world. In the scope of an European FET project, we developed Bayesian probabilistic processors that capture the results from neuromorphic computing onto robotics. The goal is to replace the digital approaches by systems directly operating on probability distributions. This has enabled us to continue to develop Bayesian hierarchical models that can be implemented in real-time. This was done concurrently with the development of artificial perception algorithms with integrated multisensory computational models mimicking cognitive systems. These include probabilistic approaches towards a new generation of algorithms to deal with uncertainty, ambiguities and conflicts inherent to the perceptual process that promote intelligent and adaptive decisions on actions in the physical world.

• Method and Apparatus for Automatic Camera Calibration using One or More Images of a Checkeboard Pattern, US9438897B2, EP2742484B1, JP5989113B2, CN103827917B
• Method for Aligning and Tracking Point Regions in Images with Radial Distortion that outputs Motion Model Parameters, Distortion Calibration, and Variation in Zoom US9367928B2, EP2904584B1
• Methods and systems for computer-aided surgery using intra-operative video acquired by a free moving camera US10499996B2
• Methods and systems for camera characterization in terms of response function, color, and vignetting under non-uniform illumination US10504239B2
• Systems and Methods for 3D Registration of Curves and Surfaces using Local Differential Information US10796499B2

• I. Felix, C. Raposo, M. Antunes, P. Rodrigues and J. P. Barreto, Towards markerless computer-aided surgery combining deep segmentation and geometric pose estimation: application in total knee arthroplasty, Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, October 2020
• C. Raposo and J. P. Barreto, “Accurate Reconstruction of Oriented 3D Points,” ECCV- European Conf. in Computer Vision, Glasgow, Aug. 2020
• P. Rodrigues, J P Barreto, and M. Antunes, “Photometric camera characterization from a single image with invariance to light intensity and vignetting.,” Computer Vision and Image Understanding, 2020.
• P. Rodrigues, M. Antunes, C. Raposo, P. Marques, F. Fonseca, and J. P. Barreto, “Deep segmentation leverages geometric pose estimation in computer-aided total knee arthroplasty.,” Healthc Technol Lett, vol. 6, no. 6, pp. 226–230, Dec. 2019.
• C. Raposo, C. Sousa, L. Ribeiro, R. Melo, F. Fonseca, J. Oliveira, P. Marques, and J. P. Barreto, “Video-based computer aided arthroscopy for patient specific reconstruction of the Anterior Cruciate Ligament,” MICCAI Medical Image Computing and Computer-Assisted Intervention, Granada Sept. 2018

• C. Raposo and J. P. Barreto, “3D Registration of Curves and Surfaces using Local Differential Information,” CVPR IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, July 2018.

• João Henriques, Rui Caseiro, Pedro Martins, Jorge Batista, High-Speed Tracking with Kernelized Correlation Filters, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.37, no.3, pp.583-596, March 2015. (IF:12.75)
• João Carreira, Rui Caseiro, Jorge Batista, Cristian Sminchisescu, Free-Form Region Description with Second-Order Pooling, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 37, no.6, pp.1177-1189, June 2015. (IF:12.75)
• Pedro Martins, Rui Caseiro, João Henriques, Jorge Batista, Bayesian Constrained Local Models Revisited, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.38, no.4, pp.704-716, April 2016. (IF:12.04)
• João Carreira, Sara Vicente, Lourdes Agapito, Jorge Batista, Lifting Object Detection Datasets into 3D, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.38, no.7, pp.1342-1355, July 2016. (IF:12.04)
• Pedro Martins, João Henriques, Jorge Batista, Gradient Shape Model, International Journal of Computer Vision, vol. 128, pp.2828–2848, Dezember 2020. (IF:11.04)
• M Tavakoli et. al., EGaIn‐Assisted Room‐Temperature Sintering of Ag Nanoparticles for Stretchable, Inkjet‐Printed, Thin‐Film Electronics, Advanced Materials 30 (29), 2018, (Impact Factor 27.5)
• P Lopes, et. al., Hydroprinted Electronics, Hydroprinted Electronics: Ultrathin Stretchable Ag-In-Ga E-Skin for Bioelectronics & Human-Machine Interaction, ACS Applied Materials and Interfaces, 2018 (Impact Factor: 8.75)
• C Leal, et. al., Untethered Disposable Health Monitoring Electronic Patches with Integrated Ag2O-Zn Battery, AgInGa Current Collector and Hydrogel Electrodes, ACS Applied Materials & Interfaces, 2019 (Impact Factor: 8.75)
• PA Lopes, et. al., “Soft Bioelectronic Stickers: Selection and Evaluation of Skin‐Interfacing Electrodes”, Advanced healthcare materials 8 (15), 1900234, 2019 (impact factor 6.27)
• DG Marques, et. al., Reliable interfaces for EGaIn multi-layer stretchable circuits and microelectronics, Lab on a Chip 19 (5), 897-906, 2019 (Impact Factor 6.8)
• J. Lobo, J. F. Ferreira, Unconventional computing for Bayesian inference, International Journal of Approximate Reasoning, Volume 88, 2017, Pages 306-308  DOI: 10.1016/j.ijar.2017.06.004
• Ferreira, J. F. ; Dias, Probabilistic Approaches for Robotic Perception". Springer Tracts in Advanced Robotics (STAR) 91, 2014, http://dx.doi.org/10.1007/978-3-319-02006-8, ISBN 978-3-319-02006-8
• J. S. Friedman, J. Droulez, P. Bessière, J. Lobo, D. Querlioz, Approximation enhancement for stochastic Bayesian inference, International Journal of Approximate Reasoning, Volume 85, June 2017, Pages 139-158, ISSN 0888-613X  DOI: 10.1016/j.ijar.2017.03.007
• P. Lanillos Pradas, J. F. Ferreira, J. Dias, A Bayesian Hierarchy for Robust Gaze Estimation in Human-Robot Interaction, Int. J. of Approximate. Reasoning, 87, 1-22, 2017 DOI: 10.1016/j.ijar.2017.04.007

• HIGHLY CITED PAPER - High-Speed Tracking with Kernelized Correlation Filters, IEEE Trans. on Pattern Analysis and Machine Intelligence (2015). Web of Knowledge Top 1% paper in Engineering;
• SPRINGER BEST PAPER AWARD (3th place) - Bruno Ferreira, Pedro M. Ferreira, Gil Pinheiro, Nelson Figueiredo, Filipe Carvalho, Paulo Menezes, and Jorge Batista, Exploring Workout Repetition Counting and Validation through Deep Learning, in A. Campilho, Fakhri Karray and Zhou Wang (Eds.) ICIAR2020 – Image Analysis and Recognition (Lecture Notes in Computer Science) Springer-Verlag, London, pp.3-15, 2020.

• Computational Vision Systems for Intelligent Transportation Systems: 2015-2020 (~45.000€ per year) : Funded by AtoBe – Mobility Technology S.A.
• PROZIS Challenge : 2019-2020 (~55.000€): Funded by PROZIS.TECH, S.A.

• Stretchtronics: Additive manufacturing of wearable bioelectronics (CMU-Portugal 2016-2020)
• WoW: Biomonitoring patches for Domiciliary Hospitalization, (CMU-Portugal 2020-2023)
• PAMI: Portuguese additive manufacturing Initiative (Infrastructure Project- 2017-2021)
• Swithome Project: In-home rehabilitation through printed insoles, (EU- EIT-Health 2018)
• IndirockNsol: 3D Printed Shoes for Diabetic foot(EU- EIT-Health 2020-2023) 

• BAMBI (Bottom-up Approaches to Machines dedicated to Bayesian Inference) project is a EU collaborative FET Project ( FP7-ICT-2013-C, project number 618024) Coordinator: Jacques Droulez, CNRS-LPPA Collège de France (France), Partners: ISIR - UPMC - Sorbonne Universités, France; Laboratoire d’Informatique de Grenoble (LIG) France; Unité Mixte de Physique CNRS/Thales (UMphi) France; Institut d’Electronique Fondamentale (IEF) France; Institute for Systems and Robotics – University of Coimbra (ISR-UC) Portugal; University of Liège (ULG) Belgium; The Hebrew University Of Jerusalem (HUJI) Israel; PROBAYES SAS, France. http://www.bambi-fet.eu Jan. 2014 - Dec. 2016
  Overall budget 3.3 M€; EU funding 2.5M€; 350 K€ funding for ISR-UC

• João Filipe Henriques : “Circulant Structures in Computer Vision”, March, 2016;