TOHOKU UNIVERSITY Researchers - Colan Zaita Jacinto Enrique

Details of the Researcher

Home

日本語 English

Colan Zaita Jacinto Enrique

Section

Graduate School of Engineering

Job title

Specially Appointed Assistant Professor(Research)

Degree

Doctor of Engineering (Nagoya University)

researchmap

https://researchmap.jp/jacinto_colan

J-GLOBAL ID

202101009544280342

e-Rad No.

40936746

ORCID

https://orcid.org/0000-0002-8833-2215

Research History 3

2026/04 - Present

Tohoku University　Graduate School of Engineering Department of Robotics　Specially Appointed Assistant Professor
2026/04 - Present

Nagoya University　Graduate School of Engineering Micro-Nano Mechanical Science and Engineering　Assistant Professor
2021/04 - 2026/03

Nagoya University　Postdoctoral Researcher

Education 2

Nagoya University　Graduate School of Engineering

2018/04 - 2021/03
Nagoya University　Graduate School of Engineering

2016/04 - 2018/03

Professional Memberships 2

The Robotics Society of Japan

2016 - Present
IEEE Robotics and Automation Society

2010 - Present

Research Interests 4

Human-Robot interfaces
Robotic Manipulation
Surgical Robotics
Assistive Robotics

Research Areas 3

Informatics / Robotics and intelligent systems /
Informatics / Mechanics and mechatronics /
Manufacturing technology (mechanical, electrical/electronic, chemical engineering) / Control and systems engineering /

Awards 13

Best Journal Paper Award

2026/07　Japan Society of Mechanical Engineers (JSME)　Kinematic analysis and experimental verification of transforming planar linkage mechanism
Best Poster Presentation Award

2026/07　40th International Congress and Exhibition on Computer Assisted Radiology and Surgery (CARS 2026)　Assessing Selective Interaction for Reliable and Trustworthy Robotic Surgical Assistance
Lab Director Award

2026/03　2025 JST AIP Challenge　アクティブ奥行き推定のための不確かさを考慮した予測制御
Outstanding Presentation Award

2026/02　第26回計測自動制御学会システムインテグレーション部門講演会 (SI2025)　衣服着脱支援ロボットにおけるプライバシーを配慮した衣服の把持位置検出システム
Best Poster Award

2025/09　2025 Conference on Robot Learning (CoRL). Workshop on Open-Source Hardware in the Era of Robot Learning　An Open-Source Platform for Dexterous Robotic Surgical Manipulation and Learning
Best Paper Award

2024/11　2024 International Symposium on Micro-NanoMechatronics and Human Science (MHS)　Embedded Image-to-Image Translation for Efficient Sim-to-Real Transfer in Learning-based Robot-Assisted Soft Manipulation
Best Paper Award Finalist

2023/08　2023 IEEE International Conference on Mechatronics and Automation (ICMA)　Manipulability maximization in constrained inverse kinematics of surgical robots
Best Paper Award

2023/08　IEEE International Workshop on Robot and Human Communication (RO-MAN)　Single Actuator Tendon Driven Two Finger Linkage Gripper with Strong Pinch and Adaptable Cylindrical Grasp
Best Paper Award Finalist

2023/03　2022 IEEE International Conference on Cyborg and Bionic Systems (CBS)　Visual Tactile Sensor Based Force Estimation for Position-Force Teleoperation
Best Paper Award

2022/11　2022 International Symposium on Micro-NanoMechatronics and Human Science (MHS)　Sensorless grip force estimation of a cable-driven robotic surgical tool based on Gaussian Process Regression
1st Place Gold Medalist in PAREDO Hackathon

2021/11　Participatory Design in Robotics for Elderly Care in Germany & Japan (PADERO)
Best Paper Award

2019/12　2019 International Symposium on Micro-NanoMechatronics and Human Science (MHS)　Usability Study on Hands-on User Interface for Neurosurgical Articulated Forceps: Joystick and Serial-link based Design
Best Presentation Award

2016/12　17th SICE System Integration Division Annual Conference　Design and Prototype of an Intuitive User Interface for Forceps with Multi-Degrees of Freedom

Show all ︎Show 5

Papers 53

Assessing selective interaction for reliable and trustworthy robotic surgical assistance Peer-reviewed

Jacinto Colan, Ana Davila, Yutaro Yamada, Kazunari Misawa, Yasuhisa Hasegawa

International Journal of Computer Assisted Radiology and Surgery　2026/09/28

DOI： 10.1007/s11548-026-03799-6 　
Prior-Guided 3D Gaussian Splatting for Data Augmentation in Endoscopic Monocular Depth Estimation Peer-reviewed

Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

2026 23rd International Conference on Ubiquitous Robots (UR)　235-240　2026/07
Publisher: IEEE
DOI： 10.1109/ur69537.2026.11626692 　
Optimizing Inverse Kinematics for Robotic Manipulators Through Concurrent Solving Peer-reviewed

Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

SN Computer Science　7　(5)　2026/05/20
Publisher: Springer Science and Business Media LLC
DOI： 10.1007/s42979-026-05067-5 　

eISSN： 2661-8907
Descriptor: A High-Fidelity 3D Dataset for Fine-Grained Monocular Depth Estimation (HFD-3D) Peer-reviewed

Jacinto Colan, Ana Davila, Yutaro Yamada, Yasuhisa Hasegawa

IEEE Data Descriptions　1-12　2026
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
DOI： 10.1109/ieeedata.2026.3700560 　

eISSN： 2995-4274
Design and Analysis of a Novel High-Extension Modular Manipulator for Dexterous and Confined-Space Operations Peer-reviewed

Jayant Unde, Yuki Wakayama, Takumi Inden, Jacinto Colan, Yasuhisa Hasegawa

IEEE Access　2026

DOI： 10.1109/ACCESS.2026.3680340 　
Adaptive Transfer Learning for Surgical Tool Presence Detection in Laparoscopic Videos Through Gradual Freezing Fine‐Tuning Peer-reviewed

Ana Davila, Jacinto Colan, Yasuhisa Hasegawa

International Journal of Imaging Systems and Technology　2025/11

DOI： 10.1002/ima.70218 　
An Open-Source Platform for Dexterous Robotic Surgical Manipulation and Learning Peer-reviewed

Jacinto Colan, Ana Davila, Yutaro Yamada, Yasuhisa Hasegawa

2025 Conference on Robot Learning (CoRL). Workshop on Open-Source Hardware in the Era of Robot Learning.　2025/10/17

DOI： 10.36227/techrxiv.176070611.16904587/v1 　
Affordance-Based Disambiguation of Surgical Instructions for Collaborative Robot-Assisted Surgery Peer-reviewed

Ana Davila, Jacinto Colan, Yasuhisa Hasegawa

2025 Conference on Robot Learning (CoRL). Workshop on Intelligent Cobodied Assistance and Robotic Empowerment (iCARE)　2025/10

More details Close

Effective human-robot collaboration in surgery is affected by the inherent ambiguity of verbal communication. This paper presents a framework for a robotic surgical assistant that interprets and disambiguates verbal instructions from a surgeon by grounding them in the visual context of the operating field. The system employs a two-level affordance-based reasoning process that first analyzes the surgical scene using a multimodal vision-language model and then reasons about the instruction using a knowledge base of tool capabilities. To ensure patient safety, a dual-set conformal prediction method is used to provide a statistically rigorous confidence measure for robot decisions, allowing it to identify and flag ambiguous commands. We evaluated our framework on a curated dataset of ambiguous surgical requests from cholecystectomy videos, demonstrating a general disambiguation rate of 60% and presenting a method for safer human-robot interaction in the operating room.
Compliant Gripper Design Concept for Safe Manipulation of Human-Worn Clothing Peer-reviewed

SICE Festival with Annual Conference (SICE FES)　2025/09
Assessing the Value of Visual Input: A Benchmark of Multimodal Large Language Models for Robotic Path Planning Peer-reviewed

Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

SICE Festival with Annual Conference (SICE FES)　2025/09

More details Close

Large Language Models (LLMs) show potential for enhancing robotic path planning. This paper assesses visual input's utility for multimodal LLMs in such tasks via a comprehensive benchmark. We evaluated 15 multimodal LLMs on generating valid and optimal paths in 2D grid environments, simulating simplified robotic planning, comparing text-only versus text-plus-visual inputs across varying model sizes and grid complexities. Our results indicate moderate success rates on simpler small grids, where visual input or few-shot text prompting offered some benefits. However, performance significantly degraded on larger grids, highlighting a scalability challenge. While larger models generally achieved higher average success, the visual modality was not universally dominant over well-structured text for these multimodal systems, and successful paths on simpler grids were generally of high quality. These results indicate current limitations in robust spatial reasoning, constraint adherence, and scalable multimodal integration, identifying areas for future LLM development in robotic path planning.
Beyond Single Models: Enhancing LLM Detection of Ambiguity in Requests through Debate Peer-reviewed

Ana Davila, Jacinto Colan, Yasuhisa Hasegawa

SICE Festival with Annual Conference (SICE FES)　2025/09

More details Close

Large Language Models (LLMs) have demonstrated significant capabilities in understanding and generating human language, contributing to more natural interactions with complex systems. However, they face challenges such as ambiguity in user requests processed by LLMs. To address these challenges, this paper introduces and evaluates a multi-agent debate framework designed to enhance detection and resolution capabilities beyond single models. The framework consists of three LLM architectures (Llama3-8B, Gemma2-9B, and Mistral-7B variants) and a dataset with diverse ambiguities. The debate framework markedly enhanced the performance of Llama3-8B and Mistral-7B variants over their individual baselines, with Mistral-7B-led debates achieving a notable 76.7% success rate and proving particularly effective for complex ambiguities and efficient consensus. While acknowledging varying model responses to collaborative strategies, these findings underscore the debate framework's value as a targeted method for augmenting LLM capabilities. This work offers important insights for developing more robust and adaptive language understanding systems by showing how structured debates can lead to improved clarity in interactive systems.
Design and Preliminary Evaluation of a Walker-Mounted Robotic System for Elderly Toilet Dressing Assistance Peer-reviewed

Jayant Unde, Taisei Urata, Shinnosuke Kamei, Yahiro Ito, Ryusei Kihara, Jacinto Colan, Yasuhisa Hasegawa

34th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)　2025/08
Human-Robot collaboration in surgery: Advances and challenges towards autonomous surgical assistants Peer-reviewed

Jacinto Colan, Ana Davila, Yutaro Yamada, Yasuhisa Hasegawa

34th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)　2025/08

More details Close

Human-robot collaboration in surgery represents a significant area of research, driven by the increasing capability of autonomous robotic systems to assist surgeons in complex procedures. This systematic review examines the advancements and persistent challenges in the development of autonomous surgical robotic assistants (ASARs), focusing specifically on scenarios where robots provide meaningful and active support to human surgeons. Adhering to the PRISMA guidelines, a comprehensive literature search was conducted across the IEEE Xplore, Scopus, and Web of Science databases, resulting in the selection of 32 studies for detailed analysis. Two primary collaborative setups were identified: teleoperation-based assistance and direct hands-on interaction. The findings reveal a growing research emphasis on ASARs, with predominant applications currently in endoscope guidance, alongside emerging progress in autonomous tool manipulation. Several key challenges hinder wider adoption, including the alignment of robotic actions with human surgeon preferences, the necessity for procedural awareness within autonomous systems, the establishment of seamless human-robot information exchange, and the complexities of skill acquisition in shared workspaces. This review synthesizes current trends, identifies critical limitations, and outlines future research directions essential to improve the reliability, safety, and effectiveness of human-robot collaboration in surgical environments.
LLM-based ambiguity detection in natural language instructions for collaborative surgical robots Peer-reviewed

Ana Davila, Jacinto Colan, Yasuhisa Hasegawa

34th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)　2025/08

More details Close

Ambiguity in natural language instructions poses significant risks in safety-critical human-robot interaction, particularly in domains such as surgery. To address this, we propose a framework that uses Large Language Models (LLMs) for ambiguity detection specifically designed for collaborative surgical scenarios. Our method employs an ensemble of LLM evaluators, each configured with distinct prompting techniques to identify linguistic, contextual, procedural, and critical ambiguities. A chain-of-thought evaluator is included to systematically analyze instruction structure for potential issues. Individual evaluator assessments are synthesized through conformal prediction, which yields non-conformity scores based on comparison to a labeled calibration dataset. Evaluating Llama 3.2 11B and Gemma 3 12B, we observed classification accuracy exceeding 60% in differentiating ambiguous from unambiguous surgical instructions. Our approach improves the safety and reliability of human-robot collaboration in surgery by offering a mechanism to identify potentially ambiguous instructions before robot action.
A conformal prediction framework for natural language ambiguity detection in surgical robot instructions Peer-reviewed

Ana Davila, Jacinto Colan, Yasuhisa Hasegawa

International Joint Conference on Neural Networks (IJCNN 2025). ForgtAI: International Workshop on Forging Trust in Artificial Intelligence　2025/07
Transfer learning optimization based on evolutionary selective fine tuning Peer-reviewed

Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

International Joint Conference on Neural Networks (IJCNN 2025). Workshop: artiFicial And bio-inspIred netwoRked intelliGence foR cOnstrained aUtoNomous Devices (FAIRGROUND).　2025/07
Kinematic analysis and experimental verification of transforming planar linkage mechanism Peer-reviewed

Jayant Unde, Yahiro Ito, Jacinto Colan, Yasuhisa Hasegawa

ROBOMECH Journal　12　(1)　2025/02/11
Publisher: Springer Science and Business Media LLC
DOI： 10.1186/s40648-025-00289-3 　

eISSN： 2197-4225

More details Close

Abstract This paper explores the kinematic characteristics of a novel transforming planar linkage mechanism characterized by a coiling and uncoiling motion pattern. Unlike traditional deployable structures that depend on scaling transformations, this mechanism utilizes a four-bar linkage system that rotates collectively to achieve the desired transformation with a single degree of freedom. The study provides a comprehensive analysis of the mechanism’s properties, highlighting its singularity points, trajectory variations, and transformation capabilities contingent on the fixed frame’s position. These properties are validated through simulations and experimental tests, supporting the analytical findings. Furthermore, a tendon-driven actuation method tailored for this mechanism is introduced and validated using a physical prototype. The results contribute significantly to understanding this innovative planar linkage mechanism, offering valuable insights for designing novel mechanical systems and applications.
IST-ROS: A flexible object segmentation and tracking framework for robotics applications Peer-reviewed

Khusniddin Fozilov, Yutaro Yamada, Jacinto Colan, Yaonan Zhu, Yasuhisa Hasegawa

SoftwareX　29　101979-101979　2025/02
Publisher: Elsevier BV
DOI： 10.1016/j.softx.2024.101979 　

ISSN： 2352-7110
Novel Articulated Lead Screw Linear Actuator Enabled by Transforming Linkage Mechanism Peer-reviewed

Jayant Unde, Jacinto Colan, Yasuhisa Hasegawa

IEEE Robotics and Automation Letters　2025/02

DOI： 10.1109/LRA.2024.3521682 　
Real-Time Motion Parallax Generation System Using Spherical Rendering and View Morphing in Unknown Environments Peer-reviewed

Sho Hatano, Jacinto Colan, Yasuhisa Hasegawa

IEEE Access　2025

DOI： 10.1109/ACCESS.2025.3641220 　
Bio-Inspired Fine-Tuning for Selective Transfer Learning in Image Classification Peer-reviewed

Ana Davila, Jacinto Colan, Yasuhisa Hasegawa

IEEE Access　2025

DOI： 10.1109/ACCESS.2025.3587524 　
Tactile Feedback in Robot‐Assisted Minimally Invasive Surgery: A Systematic Review Peer-reviewed

Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

The International Journal of Medical Robotics and Computer Assisted Surgery　2024/12

DOI： 10.1002/rcs.70019 　
Design, Modeling, and Experimental Verification of Passively Adaptable Roller Gripper for Separating Stacked Fabric Peer-reviewed

Jayant Unde, Jacinto Colan, Yasuhisa Hasegawa

IEEE Robotics and Automation Letters　2024/11

DOI： 10.1109/LRA.2024.3461550 　
Voice control interface for surgical robot assistants Peer-reviewed

Ana Davila, Jacinto Colan, Yasuhisa Hasegawa

2024 International Symposium on Micro-NanoMechatronics and Human Science, MHS 2024　2024/09/16

More details Close

Traditional control interfaces for robotic-assisted minimally invasive surgery impose a significant cognitive load on surgeons. To improve surgical efficiency, surgeon-robot collaboration capabilities, and reduce surgeon burden, we present a novel voice control interface for surgical robotic assistants. Our system integrates Whisper, state-of-the-art speech recognition, within the ROS framework to enable real-time interpretation and execution of voice commands for surgical manipulator control. The proposed system consists of a speech recognition module, an action mapping module, and a robot control module. Experimental results demonstrate the system's high accuracy and inference speed, and demonstrates its feasibility for surgical applications in a tissue triangulation task. Future work will focus on further improving its robustness and clinical applicability.
Embedded Image-to-Image Translation for Efficient Sim-to-Real Transfer in Learning-based Robot-Assisted Soft Manipulation Peer-reviewed

Jacinto Colan, Keisuke Sugita, Ana Davila, Yutaro Yamada, Yasuhisa Hasegawa

2024 International Symposium on Micro-NanoMechatronics and Human Science, MHS 2024　2024/09/16

More details Close

Recent advances in robotic learning in simulation have shown impressive results in accelerating learning complex manipulation skills. However, the sim-to-real gap, caused by discrepancies between simulation and reality, poses significant challenges for the effective deployment of autonomous surgical systems. We propose a novel approach utilizing image translation models to mitigate domain mismatches and facilitate efficient robot skill learning in a simulated environment. Our method involves the use of contrastive unpaired Image-to-image translation, allowing for the acquisition of embedded representations from these transformed images. Subsequently, these embeddings are used to improve the efficiency of training surgical manipulation models. We conducted experiments to evaluate the performance of our approach, demonstrating that it significantly enhances task success rates and reduces the steps required for task completion compared to traditional methods. The results indicate that our proposed system effectively bridges the sim-to-real gap, providing a robust framework for advancing the autonomy of surgical robots in minimally invasive procedures.
A hierarchical framework for collision avoidance in robot-assisted minimally invasive surgery Peer-reviewed

Jacinto Colan, Ana Davila, Khusniddin Fozilov, Yasuhisa Hasegawa

2024 IEEE International Conference on Cyborg and Bionic Systems (CBS 2024)　2024/09/16

More details Close

Minimally invasive surgery (MIS) procedures benefit significantly from robotic systems due to their improved precision and dexterity. However, ensuring safety in these dynamic and cluttered environments is an ongoing challenge. This paper proposes a novel hierarchical framework for collision avoidance in MIS. This framework integrates multiple tasks, including maintaining the Remote Center of Motion (RCM) constraint, tracking desired tool poses, avoiding collisions, optimizing manipulability, and adhering to joint limits. The proposed approach utilizes Hierarchical Quadratic Programming (HQP) to seamlessly manage these constraints while enabling smooth transitions between task priorities for collision avoidance. Experimental validation through simulated scenarios demonstrates the framework's robustness and effectiveness in handling diverse scenarios involving static and dynamic obstacles, as well as inter-tool collisions.
Real-time inverse kinematics for robotic manipulation under remote center-of-motion constraint using memetic evolution Peer-reviewed

Ana Davila, Jacinto Colan, Yasuhisa Hasegawa

Journal of Computational Design and Engineering　11　(3)　248-264　2024/06/01

DOI： 10.1093/jcde/qwae047 　

ISSN： 2288-4300

eISSN： 2288-5048
Comparison of fine-tuning strategies for transfer learning in medical image classification Peer-reviewed

Ana Davila, Jacinto Colan, Yasuhisa Hasegawa

Image and Vision Computing　2024/06

DOI： 10.1016/j.imavis.2024.105012 　

ISSN： 0262-8856
Multimodal semi-supervised learning for online recognition of multi-granularity surgical workflows Peer-reviewed

Yutaro Yamada, Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

International Journal of Computer Assisted Radiology and Surgery　2024/04/01

DOI： 10.1007/s11548-024-03101-6 　
Latent regression based model predictive control for tissue triangulation Peer-reviewed

Songtao Liu, Jacinto Colan, Yaonan Zhu, Taisuke Kobayashi, Kazunari Misawa, Masaru Takeuchi, Yasuhisa Hasegawa

Advanced Robotics　2024/03/03

DOI： 10.1080/01691864.2024.2324303 　
Variable Step Sizes for Iterative Jacobian-Based Inverse Kinematics of Robotic Manipulators Peer-reviewed

Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

IEEE Access　12　87909-87922　2024

DOI： 10.1109/ACCESS.2024.3418206 　

eISSN： 2169-3536
Development of a compliant gripper for safe robot-assisted trouser dressing-undressing Peer-reviewed

Jayant Unde, Takumi Inden, Yuki Wakayama, Jacinto Colan, Yaonan Zhu, Tadayoshi Aoyama, Yasuhisa Hasegawa

Advanced Robotics　2024

DOI： 10.1080/01691864.2024.2376024 　

ISSN： 0169-1864

eISSN： 1568-5535
Endoscope Automation Framework with Hierarchical Control and Interactive Perception for Multi-Tool Tracking in Minimally Invasive Surgery Peer-reviewed

Khusniddin Fozilov, Jacinto Enrique Colan Zaita, Ana Davila, Kazunari Misawa, Jie Qiu, Yuichiro Hayashi, Kensaku Mori, Yasuhisa Hasegawa

Sensors　2023/12/16

DOI： 10.3390/s23249865 　
A Concurrent Framework for Constrained Inverse Kinematics of Minimally Invasive Surgical Robots Peer-reviewed

Jacinto Colan, Ana Davila, Khusniddin Fozilov, Yasuhisa Hasegawa

Sensors　23　(6)　2023/03

DOI： 10.3390/s23063328 　

ISSN： 1424-8220
Gradient-Based Fine-Tuning Strategy for Improved Transfer Learning on Surgical Images Peer-reviewed

Ana Davila, Jacinto Colan, Yasuhisa Hasegawa

2023 International Symposium on Micro-NanoMechatronics and Human Science, MHS 2023　2023

DOI： 10.1109/MHS59931.2023.10510130 　
Enhancing Gradient-Based Inverse Kinematics with Dynamic Step Sizes Peer-reviewed

Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

2023 International Symposium on Micro-NanoMechatronics and Human Science, MHS 2023　2023

DOI： 10.1109/MHS59931.2023.10510153 　
Single Actuator Tendon Driven Two Finger Linkage Gripper with Strong Pinch and Adaptable Cylindrical Grasp Peer-reviewed

Jayant Unde, Jacinto Colan, Yaonan Zhu, Tadayoshi Aoyama, Yasuhisa Hasegawa

IEEE International Workshop on Robot and Human Communication, RO-MAN　119-125　2023

DOI： 10.1109/RO-MAN57019.2023.10309512 　

ISSN： 1944-9445

eISSN： 1944-9437
Manipulability maximization in constrained inverse kinematics of surgical robots Peer-reviewed

Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

2023 IEEE International Conference on Mechatronics and Automation, ICMA 2023　569-574　2023

DOI： 10.1109/ICMA57826.2023.10215986 　
Constrained Motion Planning for a Robotic Endoscope Holder Based on Hierarchical Quadratic Programming Peer-reviewed

Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

2023 8th International Conference on Control and Robotics Engineering, ICCRE 2023　198-203　2023

DOI： 10.1109/ICCRE57112.2023.10155579 　
Task Segmentation Based on Transition State Clustering for Surgical Robot Assistance Peer-reviewed

Yutaro Yamada, Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

2023 8th International Conference on Control and Robotics Engineering, ICCRE 2023　260-264　2023

DOI： 10.1109/ICCRE57112.2023.10155581 　
Toward Autonomous Robotic Minimally Invasive Surgery: A Hybrid Framework Combining Task-Motion Planning and Dynamic Behavior Trees Peer-reviewed

Khusniddin Fozilov, Jacinto Colan, Kosuke Sekiyama, Yasuhisa Hasegawa

IEEE Access　11　91206-91224　2023

DOI： 10.1109/ACCESS.2023.3308619 　

eISSN： 2169-3536
OpenRST: An Open Platform for Customizable 3D Printed Cable-Driven Robotic Surgical Tools Peer-reviewed

Jacinto Colan, Ana Davila, Yaonan Zhu, Tadayoshi Aoyama, Yasuhisa Hasegawa

IEEE Access　11　6092-6105　2023

DOI： 10.1109/ACCESS.2023.3236821 　

eISSN： 2169-3536
A Review on Tactile Displays for Conventional Laparoscopic Surgery Peer-reviewed

Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

Surgeries　3　(4)　334-346　2022/11/25
Publisher: MDPI AG
DOI： 10.3390/surgeries3040036 　

eISSN： 2673-4095

More details Close

Laparoscopic surgery (LS) is a minimally invasive technique that offers many advantages over traditional open surgery: it reduces trauma, scarring, and shortens recovery time. However, an important limitation is the loss of tactile sensations. Although some progress has been made in robotic-assisted minimally invasive surgery (RMIS) setups, RMIS is still not widely accessible. This review aims to identify which tactile display technologies have been proposed and experimentally validated for the restoration of tactile sensations during conventional laparoscopic surgical tasks. We conducted a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We identified relevant articles published over the past 10 years through a search on Web of science, Scopus, IEEE Xplore Digital, and PubMed repositories. A total of 143 articles met the inclusion criteria and 24 were included in the final review. From the reviewed articles, we classified the proposed tactile displays into two categories based on the use of skin contact: (i) skin tactile displays, which include vibrotactile, skin-indentation, and grip-feedback devices, and (ii) non-contact tactile displays based on visualization tools. This survey aims to contribute to further research in the area of tactile displays for laparoscopic surgery by providing a better understanding of the current state of the art and identifying the remaining challenges.
Cutaneous Feedback Interface for Teleoperated In-Hand Manipulation Peer-reviewed

Yaonan Zhu, Jacinto Colan, Tadayoshi Aoyama, Yasuhisa Hasegawa

2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)　2022-October　605-611　2022/10/23
Publisher: IEEE
DOI： 10.1109/iros47612.2022.9982247 　

ISSN： 2153-0858

eISSN： 2153-0866
Sensorless grip force estimation of a cable-driven robotic surgical tool based on Gaussian Process Regression Peer-reviewed

Jacinto Colan, Yasuhisa Hasegawa

2022 International Symposium on Micro-NanoMechatronics and Human Science, MHS 2022　2022
Intension Reading based Task Outcome Prediction for Operability Improvement of Time-Delayed Teleoperation System Peer-reviewed

Keisuke Fusano, Yaonan Zhu, Jacinto Enrique Colan Zaita, Tadayoshi Aoyama, Yasuhisa Hasegawa

2022 International Symposium on Micro-NanoMechatronics and Human Science, MHS 2022　2022

DOI： 10.1109/MHS56725.2022.10092073 　
Visual Tactile Sensor Based Force Estimation for Position-Force Teleoperation Peer-reviewed

Yaonan Zhu, Shukrullo Nazirjonov, Bingheng Jiang, Jacinto Colan, Tadayoshi Aoyama, Boris Belousov, Kay Hansel, Jan Peters

2022 IEEE INTERNATIONAL CONFERENCE ON CYBORG AND BIONIC SYSTEMS, CBS　49-52　2022

DOI： 10.1109/CBS55922.2023.10115342 　
Optimization-Based Constrained Trajectory Generation for Robot-Assisted Stitching in Endonasal Surgery Peer-reviewed

Jacinto Colan, Jun Nakanishi, Tadayoshi Aoyama, Yasuhisa Hasegawa

Robotics　10　(1)　27-27　2021/02/01
Publisher: {MDPI} {AG}
DOI： 10.3390/robotics10010027 　

eISSN： 2218-6581
Bidirectional Tether Less Soft Actuator with Expeditious Position Control Mechanism Peer-reviewed

Jayant Unde, Jacinto Colan, Yasuhisa Hasegawa

2021 International Symposium on Micro-NanoMechatronics and Human Science, MHS 2021　2021

DOI： 10.1109/MHS53471.2021.9767140 　
A Cooperative Human-Robot Interface for Constrained Manipulation in Robot-Assisted Endonasal Surgery Peer-reviewed

Jacinto Colan, Jun Nakanishi, Tadayoshi Aoyama, Yasuhisa Hasegawa

Applied Sciences　10　(14)　4809-4809　2020/07/13
Publisher: {MDPI} {AG}
DOI： 10.3390/app10144809 　

eISSN： 2076-3417
Usability Study on Hands-on User Interface for Neurosurgical Articulated Forceps: Joystick and Serial-link based Design Peer-reviewed

Daisuke Uozumi, Jacinto E. Colan Zaita, Jun Nakanishi, Tadayoshi Aoyama, Yasuhisa Hasegawa

2019 INTERNATIONAL SYMPOSIUM ON MICRO-NANOMECHATRONICS AND HUMAN SCIENCE (MHS)　2019
Development of a Precision-grip based Interface for 4-DoF Articulated Forceps Peer-reviewed

Keisuke Ohara, Jacinto E. Colan Zaita, Daisuke Uozumi, Tadayoshi Aoyama, Jun Nakanishi, Yasuhisa Hasegawa

2018 INTERNATIONAL SYMPOSIUM ON MICRO-NANOMECHATRONICS AND HUMAN SCIENCE (MHS)　2018

DOI： 10.1109/MHS.2018.8886921 　

ISSN： 2474-378X

eISSN： 2474-3798
A Concept of a User Interface Capable of Intuitive Operation of 4-DoF Articulated Forceps Peer-reviewed

Jacinto E. Colan Zaita, Jun Nakanishi, Keisuke Ohara, Tadayoshi Aoyama, Yasuhisa Hasegawa

2017 INTERNATIONAL SYMPOSIUM ON MICRO-NANOMECHATRONICS AND HUMAN SCIENCE (MHS)　2018-January　1-3　2017

DOI： 10.1109/MHS.2017.8305268 　

ISSN： 2474-378X

eISSN： 2474-3798

Show all ︎Show first 5

Misc. 14

衣服着脱支援ロボットにおけるプライバシーを配慮した衣服の把持位置検出システム

小林昂平, Jayant Unde, 浦田大誠, Jacinto Colan, 長谷川泰久

第26回計測自動制御学会システムインテグレーション部門講演会 (SI2025)　2026/02
マルチモーダル・マルチタスク学習を用いた協働組織切除における複数器具ジェスチャー認識

山田祐太郎, Colan Jacinto, Davila Ana, 三澤一成, 長谷川泰久

第34回日本コンピュータ外科学会大会　2025/11
Multimodal Semi-Supervised Learning for Online Recognition of Multigranularity Surgical Workflows Peer-reviewed

Yamada, Y., Colan, J., Davila, A., Hasegawa, Y.

The 15th International Conference on Information Processing in Computer-Assisted Interventions (IPCAI)　2024
Autonomous Orientation Control of Forceps Based on Real-Time Action Segmentation in Robotic Surgery

Yamada, Y., Colan, J., Davila, A., Hasegawa, Y.

2024 IEEE International Conference on Robotics and Automation (ICRA)　2024
ロボット手術におけるマルチモーダル自己教師あり表現学習に基づくジェスチャー及びスキル認識

山田祐太郎, Colan Jacinto, Davila Ana, 長谷川泰久

日本機械学会ロボティクス・メカトロニクス講演会　2024
医師からの信頼性向上のための確率分布出力に基づいた追加学習

杉田圭祐, Jacinto E. Colan Zaita, 青山忠義, 竹内大, 長谷川泰久

日本機械学会ロボティクス・メカトロニクス講演会　2023
Environment Recognition Based Shared Control for Operability Improvement of Time-Delayed Teleoperation System

房野敬介, ZHU Yaonan, COLAN Jacinto, 青山忠義, 長谷川泰久

日本機械学会ロボティクス・メカトロニクス講演会講演論文集(CD-ROM)　2022　2022

ISSN： 2424-3124
Improving Dexterity for a Hands-on User Interface to Operate Articulated Forceps via Index Finger Movement with Seizing Grip

魚住大輔, COLAN ZAITA Jacinto E., 長谷川泰久, 青山忠義, 中西淳

日本機械学会ロボティクス・メカトロニクス講演会講演論文集(CD-ROM)　2020　2020

ISSN： 2424-3124
スマートアーム

長谷川泰久, Jacinto Colan, 魚住大輔, 青山忠義, 中西淳

システム／制御／情報　63　(10)　2019/10

DOI： 10.11509/isciesci.63.10_424 　

ISSN： 0916-1600
バイオニックヒューマノイドスマートアーム-ユーザインタフェース

長谷川泰久, COLAN ZAITA Jacinto E., 魚住大輔, 青山忠義, 中西淳

システム/制御/情報　63　(10)　2019

ISSN： 0916-1600
鉗子直接操作機構と鉗子先端屈曲の同時操作を可能とする手の動きを活用した操作インターフェースの開発

魚住大輔, 大原圭祐, ZAITA Jacinto E. Colan, 青山忠義, 中西淳, 長谷川泰久

日本機械学会ロボティクス・メカトロニクス講演会講演論文集(CD-ROM)　2019　2019

ISSN： 2424-3124
多自由度能動鉗子操作用ピンチ操作型インターフェースの試作

大原圭祐, ZAITA Jacinto E. Colan, 魚住大輔, 青山忠義, 中西淳, 長谷川泰久

日本ロボット学会学術講演会予稿集(CD-ROM)　36th　2018
A Preliminary Study on Cooperative Force Control based Guidance for Accurate Pre-insertion Positioning of Surgical Instruments

COLAN ZAITA Jacinto E., SATO Yuichiro, NAKANISHI Jun, HASEGAWA Yasuhisa

日本機械学会ロボティクス・メカトロニクス講演会講演論文集(CD-ROM)　2017　2017

ISSN： 2424-3124
多自由度鉗子の直感的操作可能なユーザインタフェースの設計・試作

佐藤雄一朗, ZAITA Jacinto E. Colan, 中西淳, 長谷川泰久

計測自動制御学会システムインテグレーション部門講演会(CD-ROM)　17th　2016

Show all ︎Show first 5

Research Projects 4

Language-guided learning for generalizable manipulation of deformable objects using multimodal adaptive policies

Colan Jacinto

Offer Organization: 日本学術振興会

System: 科学研究費助成事業

Category: 若手研究

Institution: 名古屋大学

2025/04/01 - 2028/03/31
Uncertainty-driven predictive control for active depth estimation

Colan Jacinto

System: AIPチャレンジプログラム

2025/07 - 2026/03
Learning in-hand manipulation for a compliant underactuated gripper with interactive human supervision

Colan Jacinto

Offer Organization: Japan Society for the Promotion of Science

System: Grants-in-Aid for Scientific Research

Category: Grant-in-Aid for Early-Career Scientists

Institution: Nagoya University

2022/04/01 - 2024/03/31

More details Close

Our project has focused on developing robotic grippers and hands for learning manipulation tasks. We proposed several innovative designs, including a cable-driven gripper with decoupled joints for effective grasping force estimation and a variation of anthropomorphic robotic fingers using linkage mechanisms to increase rigidity and pinch force. Additionally, we developed a three-finger underactuated robotic hand with compliant joints and cable-driven actuation, where each finger is actuated by a single motor managing both flexion and abduction movements. Significant progress has been made in enhancing the control mechanisms and interaction capabilities of this new hand gripper. We created a simulation environment to train in-hand manipulation of small objects, implementing and validating several reinforcement learning algorithms. Furthermore, we implemented Image-to-Image Contrastive Unpaired Translation (CUT) for realistic policy training inputs to facilitate sim-to-real transfer.
Reliable online tool-tissue force estimation for robot-assisted surgical systems

Colan Jacinto

System: AIPチャレンジプログラム

2022/07 - 2023/03