Matching, a task to optimally assign limited resources under constraints, is a fundamental technology for society. The task potentially has various objectives, conditions, and constraints; however, the efficient neural network architecture for matching is underexplored. This paper proposes a novel graph neural network (GNN), \textit{WeaveNet}, designed for bipartite graphs. Since a bipartite graph is generally dense, general GNN architectures lose node-wise information by over-smoothing when deeply stacked. Such a phenomenon is undesirable for solving matching problems. WeaveNet avoids it by preserving edge-wise information while passing messages densely to reach a better solution. To evaluate the model, we approximated one of the \textit{strongly NP-hard} problems, \textit{fair stable matching}. Despite its inherent difficulties and the network's general purpose design, our model reached a comparative performance with state-of-the-art algorithms specially designed for stable matching for small numbers of agents.

We present NeuralLabeling, a labeling approach and toolset for annotating a scene using either bounding boxes or meshes and generating segmentation masks, affordance maps, 2D bounding boxes, 3D bounding boxes, 6DOF object poses, depth maps and object meshes. NeuralLabeling uses Neural Radiance Fields (NeRF) as renderer, allowing labeling to be performed using 3D spatial tools while incorporating geometric clues such as occlusions, relying only on images captured from multiple viewpoints as input. To demonstrate the applicability of NeuralLabeling to a practical problem in robotics, we added ground truth depth maps to 30000 frames of transparent object RGB and noisy depth maps of glasses placed in a dishwasher captured using an RGBD sensor, yielding the Dishwasher30k dataset. We show that training a simple deep neural network with supervision using the annotated depth maps yields a higher reconstruction performance than training with the previously applied weakly supervised approach.

Representing crystal structures of materials to facilitate determining them via neural networks is crucial for enabling machine-learning applications involving crystal structure estimation. Among these applications, the inverse design of materials can contribute to explore materials with desired properties without relying on luck or serendipity. We propose neural structure fields (NeSF) as an accurate and practical approach for representing crystal structures using neural networks. Inspired by the concepts of vector fields in physics and implicit neural representations in computer vision, the proposed NeSF considers a crystal structure as a continuous field rather than as a discrete set of atoms. Unlike existing grid-based discretized spatial representations, the NeSF overcomes the tradeoff between spatial resolution and computational complexity and can represent any crystal structure. We propose an autoencoder of crystal structures that can recover various crystal structures, such as those of perovskite structure materials and cuprate superconductors. Extensive quantitative results demonstrate the superior performance of the NeSF compared with the existing grid-based approach.

This paper revisits datasets and evaluation criteria for Symbolic Regression (SR), specifically focused on its potential for scientific discovery. Focused on a set of formulas used in the existing datasets based on Feynman Lectures on Physics, we recreate 120 datasets to discuss the performance of symbolic regression for scientific discovery (SRSD). For each of the 120 SRSD datasets, we carefully review the properties of the formula and its variables to design reasonably realistic sampling ranges of values so that our new SRSD datasets can be used for evaluating the potential of SRSD such as whether or not an SR method can (re)discover physical laws from such datasets. We also create another 120 datasets that contain dummy variables to examine whether SR methods can choose necessary variables only. Besides, we propose to use normalized edit distances (NED) between a predicted equation and the true equation trees for addressing a critical issue that existing SR metrics are either binary or errors between the target values and an SR model's predicted values for a given input. We conduct benchmark experiments on our new SRSD datasets using various representative SR methods. The experimental results show that we provide a more realistic performance evaluation, and our user study shows that the NED correlates with human judges significantly more than an existing SR metric.

Accurate and robust 3D measurement is widely required by industrial robot usage; however, 3D measurement under complex lighting conditions such that scenes include metallic objects or semi-transparent objects is still a difficult problem. We tackled this problem by using the Light Transport Matrix (LTM) sparse estimation. LTM is one of the representations of reflection on a projector-camera system, and by using LTM and epipolar geometry, the direct components of the reflection can be extracted. We proposed an extension of LTM estimation which enables sparse estimation to success under saturated condition. Our method utilizes Alternating Direction Method of Multipliers (ADMM) for both of the norm function and the saturation function. The key idea is to separate saturated part of the observation model from the original i\ minimization formulation. We also demonstrate that our measurement method performs well for robot visions by integrating to an actual robot task.

© 2019 IEEE. The use of robots in automated factories requires accurate bin-picking to ensure that objects are correctly identified and selected. In the case of objects with multiple reflections from their surfaces, this is a challenging task. We attempted to address this problem by developing a 3D measurement method based on a Light Transport Matrix (LTM), which can be applied to shiny objects or semi-transparent objects. The study presented herein evaluates the accuracy of the proposed method as well as the method for 3D pose estimation we previously reported, by examining a bin-picking task, which is a well-known robot application in factory automation. There is considerable demand for automated bin-picking systems for general objects. However, in the use of cost-effective measurement systems, some objects such as shiny metallic objects continue to prove problematic in terms of bin-picking because of the difficulty to measure their shapes accurately. Our 3D measurement method uses only a projector-camera system; thus, it is cost-effective, and it does not require any special optical system. It is based on fast LTM sparse estimation. We previously demonstrated that this approach can measure the 3D shape of metallic objects and showed that our pose estimation method is applicable to bin-picking. However, we did not verify its accuracy with the application of 3D robot vision. In this study, we integrate these two methods, and demonstrate that our 3D measurement method, in combination with our pose estimation work, can successfully accomplish bin-picking tasks involving shiny metallic industrial objects. Ultimately, we achieved 100 [%] picking success for 5 scenes including 15 pieces. We concluded that our proposed methods are sufficiently accurate to carry out bin-picking tasks in automated factory environments.

© 2019 IEEE. In this paper, we introduce a one-shot learning approach to extend learned point cloud gesture categories into recognizing newly introduced categories after training the original model. This approach is based on learning the discrimination between gesture classes of point cloud data, making it possible to recognize new classes without retraining the original deep neural network (DNN) model. We develop a temporal variant of the PointNet model referred to as Temporal PointNet or TPoinNet, which consumes a sequence of raw point clouds from a low-cost depth sensor and outputs the class of the sequence. We use a multitask strategy where the model learns the classification of a point cloud sequence and a Euclidean space discrimination between the classified sequence and another sequence of a different class. The new model is able to classify and map the point clouds sequence inputs into a Euclidean space where the distances between the gesture sequences correspond to the gesture similarities. We present results on a point cloud dataset and the MSR Action 3D dataset showing the discrimination of new gesture categories with a high precision.

Copyright © 2018 Inderscience Enterprises Ltd. We introduce homogeneous light transport (HLT) matrix for efficient 3D measurements. Which is an extension of the light transport (LT) matrix. The LT Matrix is a model of light transportation between a projector and a camera. It describes weights for all combinations of the projector pixels and the camera pixels. The HLT Matrix includes background lights information with LT Matrix. By using the HLT Matrix, we can estimate LT Matrix without backgrounds measurement. In this paper, we demonstrate that 3D points can be measured by using '1-pixel projection' and considering the epipolar geometry, for measurement targets which are difficult to measure by traditional methods. This measurement is the same as the measurement using the LT Matrix approach. We use a compressive sensing method for estimation to reduce the number of projection and capture.

© 2018 IEEE. 3D measurement of target objects characterized by specular reflection or subsurface scatterings cannot be measured by traditional 3D measurement methods because these targets have multiple light paths that make it difficult to determine the unique surface. We define these objects as complex light reflection objects. In this case, 3D measurement methods based on Light Transport (LT) Matrix estimation may be a solution to measure these complex light reflection objects, because LT Matrix captures every light path, and we can identify all 3D points on the target shape by using LT Matrix. However, these methods either provide low resolution results, or they are too slow for use in robot vision in practice. In this paper, we suppress the computational cost of LT Matrix estimation by dividing LT Matrix estimation into multi-scale. The proposed method reduces the number of candidate combinations between camera pixels and projector pixels greatly by using the information given by low resolution observations. The proposed algorithm allows high resolution measurement of the LT Matrix very efficiently. Furthermore, careful implementation of our method by using a sparse matrix representation achieves memory efficiency. We evaluated our method by measuring 3D points for a 256 × 256 resolution projector and camera system, which is an LT matrix 4096 times larger than that developed in our previous study [1] and 100 times faster than our naïve implementation of [2].

© 2018. The copyright of this document resides with its authors. The Light Transport Matrix (LTM) is a model of the light ray propagation between a projector and a camera. In case of LTM measurement, sparse estimations are often used. They assume the linearity between the projector and camera intensities. Sparse estimation requires multiple projector pixels to be irradiated together. Since multiple projector pixels are irradiated, the camera captures both the direct and global illumination. When the intensity of the illumination received by a camera pixel is higher than the threshold, camera intensity is clipped to the threshold. The camera intensities can be saturated, even if the LTM elements are not saturated, because of the global illumination. This saturation breaks the assumption of sparse estimation and causes the estimated result to be inaccurate. We propose a new sparse estimation algorithm “Saturation ADMM,” which estimates the LTM under conditions in which camera images are saturated because of global illumination. We used numerical simulation and real scene measurement experiments to prove the ability of the proposed method to accurately estimate the LTM under saturated conditions.

We introduce Homogeneous Light Transport (HLT) Matrix for efficient 3D measurements. Which is an extension of the Light Transport (LT) matrix. The LT Matrix is a model of light transportation between a projector and a camera. It describes weights for all combinations of the projector pixels and the camera pixels. The HLT Matrix includes background lights information with LT Matrix. By using the HLT Matrix, we can estimate LT Matrix without backgrounds measurement. In the traditional methods, it is assumed that the light ray reflects only once in the scene. However, in the actual scenes, it may reflect multiple times due to specular reflections or subsurface scatterings thus 3D measurement may become difficult in some cases. The LT Matrix can express multiple times reflection. In this paper, we demonstrate that 3D points can be measured by using '1-pixel projection' and considering the epipolar geometry, for measurement targets which are difficult to measure by traditional methods. This measurement is the same as the measurement using the LT Matrix approach. We use a compressive sensing method for estimation to reduce the number of projection and capture.

Active stereo vision based on the use of a projector and a camera is one of the major 3D measurement methods. Active stereo vision is a technique that is widely used because its ability to measure 3D shapes is robust to changes in ambient light in the environment. The technique involves the projection of patterns known as structured light from a projector, after which the resulting images are captured by a camera. However, active stereo vision is unable to perform effective measurements in scenes that include specular reflections or subsurface scattering. Although various methods have been proposed to solve this problem, no pattern capable of adapting to measuring varying optical properties has been proposed to date. Thus, an approach that enables projection patterns to be designed is not readily available. This led us to propose a new concept named "Feedback Projection for 3D Measurement" for adaptive measurement in a variety of scenes. In this concept, projection patterns are generated dynamically by adapting to the lighting scene to achieve high-speed and high-accuracy 3D measurements of scenes characterized by complex light reflectance. In the research presented in this paper, we resolve the 3D measurement problem as a problem of estimating the Light Transport (LT) matrix. Our proposed method consists of estimating the LT matrix and generating the projection pattern by using feedback projection. A numerical simulation is used to verify the concept of the proposed method.