Lee et al. (2019a) presented conditional generative adversarial nets (cGANs) to generate visual data from tactile sensory input and vice-versa. Different variations of the experiments showed that the target robot consistently achieved better than chance accuracy. The results showed that KEMA could transfer existing knowledge to the target robot, Attention in transformers accurately classifying all unseen objects. A possible disadvantage of the proposed methods is the need for both the source data and (portion of) the target data.
Bhattacharjee et al. (2018) combined haptic information (i.e., force + motion) with thermal sensing to recognise objects in daily living environments. The collected data from multiple grasps was classified using k-means, quadratic discriminant analysis (QDA), k-nearest neighbours (kNN), and support vector machines (SVM). The containers were squeezed, and both pressure and perceived visual deformation were used for classification. Hence, some examples are shown to provide a glance into the state of the art of multimodal object perception applications. Such initiatives would also help to cope with the diverse number of robot embodiments, i.e., different sensors and actuators, which hinders progress on multimodal object perception. The reduced number and small size of public datasets for multimodal object perception motivates the study of transfer learning from visual object recognition to tactile object recognition.
Finally, hybrid data approaches use a shared modality or dataset to achieve the transfer (Baltrušaitis et al., 2019). In contrast, non-parallel data approaches can use data from a different dataset with overlapping classification categories. Co-learning is task-independent and could be used in fusion, translation, and alignment models (Baltrušaitis et al., 2018). Co-learning is particularly useful when data for Large language models some modality is limited, and information from a different modality can be used to aid training by exploiting complementary information across modalities. Co-learning is described as a more general form of transfer learning at the level of representation or inference.
III-A Modeling Point Clouds using Sparse 3D Encoder
Whereas generation from tactile-to-visual achieved similarities ranging from 50% to 90%. The results showed that visual-to-tactile generation achieves a similarity of around 90%. They used the ViTac https://aicurated.com/ dataset of cloth textures, which consists of 100 different pieces of fabric.
In contrast, GFK focuses on geometric and statistical changes from the source domain to the target domain. Thirdly, transfer learning three methods based on dimensionality reduction were tested, namely, transfer component analysis (TCA), subspace alignment (SA), and geodesic flow kernel (GFK). In particular, equalizing partiality allowed to filter out the noise and reconstruct missing portions of the surface, and uniforming density was used to downsample the point density while creating a more uniform point density. Firstly, a visuo-tactile common representation based on point clouds was preprocessed to obtain similarly sized representations.
2025 Top Article – The Ultimate Guide to Depth Perception and 3D Imaging Technologies – Robotics Tomorrow
2025 Top Article – The Ultimate Guide to Depth Perception and 3D Imaging Technologies.
Posted: Wed, 24 Dec 2025 14:04:44 GMT source
The final challenge described by Baltrušaitis et al. (2018) is co-learning. This kind of setup has been used extensively with two substreams focusing on processing visual and haptic inputs separately. In midst-mapping fusion, the feature descriptors are provided to https://aicurated.com/ai-applications/realsense-spins-out-of-intel-as-3d-perception-demand-grows/ the model separately. While this strategy is simplistic and hence easy to implement, the disadvantage is that each modality’s impact on the result is fixed as it depends on the respective feature vector’s size instead of its statistical relevance.
- In contrast, GFK focuses on geometric and statistical changes from the source domain to the target domain.
- A possible disadvantage of the proposed methods is the need for both the source data and (portion of) the target data.
- Thirdly, transfer learning three methods based on dimensionality reduction were tested, namely, transfer component analysis (TCA), subspace alignment (SA), and geodesic flow kernel (GFK).
- In particular, equalizing partiality allowed to filter out the noise and reconstruct missing portions of the surface, and uniforming density was used to downsample the point density while creating a more uniform point density.
- Bhattacharjee et al. (2018) combined haptic information (i.e., force + motion) with thermal sensing to recognise objects in daily living environments.
