Challenges

The objective is to estimate the contact state, which includes the contact locations on distinct links ($\mathbf{r}_i \in \mathbb{R}^3$), the corresponding 3D force vectors ($\mathbf{F}_i \in \mathbb{R}^3$), and the number of contacts (up to two), using proprioceptive sensors. This is an inverse problem characterized by:

• Multi-modality (singularity): while the forward mapping (from contact to sensor reading) is deterministic, the inverse mapping is ambiguous; distinct contact states may correspond to identical sensor measurements, so the probability distribution of possible solutions becomes multi-modal.
• Temporal dependency: leveraging past estimates provides a prior context for inferring the current state, for both discrete evolution (e.g., a new contact occurs) and continuous evolution (e.g., a contact slides).
• Contact event detection: determining the number of active contacts requires identifying discrete transitions, specifically the addition or removal of contacts.

Examples of key challenges in the contact estimation problem. (a) Multi-modality: Two distinct contact forces ($F_{1a}$ and $F_{1b}$) result in identical observations. (b) Temporal dependency: Current contact states depend on history, involving discrete changes in the number of contacts (e.g., when a new contact $F_2$ occurs, $t-2 \rightarrow t-1$) and continuous updates in locations or forces (e.g., when an existing contact $F_1$ slides, $t-1 \rightarrow t$). (c) Contact event detection: Determining the number of active contacts relies on identifying the exact timing of a transition and whether a contact is added or removed.

Limitations of existing approaches

While PF-based approaches are well-suited for addressing temporal dependency through recursive belief updates, they often fail to handle multi-modality due to particle starvation. In contrast, learning-based approaches excel at estimating multi-modal distributions; however, these feed-forward approaches are typically non-recursive and trained on temporally independent samples, thus lacking a native mechanism for leveraging temporal dependency. As for contact event detection, PFs rely on hard-to-tune thresholds to identify discrete transitions.

Particle starvation. (a) The results of the QP are visualized over the entire surface for a single contact ($F_1$) on link 2, where high-belief (low QP error) regions are observed on both links 1 and 2. (b) The conventional PF algorithm populates the particle set on only a single link in each execution (Trial 1 and Trial 2).

Particle drift. (a) The first particle set converges to the first contact point. (b) After the second contact ($F_2$) occurs, the delayed initialization of the second particle set causes the existing set to drift away from the first contact point.

Key idea: mixture of flow matching and particle filter

The proposed method samples particles from two complementary proposal distributions. Top: The PF's temporal prediction involves sampling from the previous time step, random position updates, and importance weighting. Bottom: The flow matching samples are transported along a probability path ($p^{[0]} \rightarrow p^{[1]}$) to represent the contact points. Right: These proposals are merged to integrate temporal context and multi-modality, forming the combined distribution for the subsequent recursive prediction.

To formally implement the mixture of two proposal distributions, we adopt the dual-sampling concept. The output of the flow matching model serves as a sample from the dual-proposal distribution, $\hat{\mathbf{X}}_t \sim p_\theta(\mathbf{X}_t \mid \mathbf{O}_t) \approx \bar{q}(\mathbf{r}_t)$. The weight of a dual particle is computed by evaluating the Gaussian probability density of the motion model:

Intuitively, dual particles that are more consistent with the previous step's particle set receive higher weights. By preserving the PF's recursive structure, the proposed method captures temporal dependency, while the dual-sampling step enables the direct generation of samples in high-belief regions, even where the particles fail to populate.

Technical components

1. Data representation and transformer-based architecture

(a) The observation for flow matching is defined as a sequence where each link's pose $\mathbf{T}_l$ is augmented with the virtual wrench $\mathcal{F}_{v,l}$. (b) $\mathcal{F}_{v,l}$ is obtained by projecting the spatially distributed external base wrench and the first $l$ elements of the external joint torque onto the corresponding frame $\{L_l\}$ via the adjoint transformation. A transformer encoder processes this sequence to extract cross-modal correlations.

The training sample $\mathbf{X}^{[1]} \in \mathbb{R}^{N_s \times 6}$ is constructed based on the number of contact points $n_c$: each row is $(\mathbf{r}_1, \mathbf{r}_1)$ for $n_c = 1$ and $(\mathbf{r}_1, \mathbf{r}_2)$ for $n_c = 2$. This paired-point representation captures $n_c$, thereby eliminating the need for a separate classifier.

2. Flow matching with few-step inference

Comparison of generative processes implemented with our architecture, using (a) Flow Matching and (b) DDIM. (•-1): Sample trajectories over ODE time steps (128-step integration). (•-2): Accelerated generation paths with 2-step integration. (•-3): Resulting samples via the 2-step integration. The sample trajectories generated by Flow Matching are markedly more linear than the curved paths produced by DDIM, which preserves high consistency between the results of 2-step and 128-step ODE integration.

2-step Flow Matching achieves M-RMSE values of 0.17 cm (single-contact) and 0.72 cm (dual-contact), competitively matching or even surpassing methods utilizing 16 ODE steps, while being eight times faster (2.57 ms). The flow matching inference is executed asynchronously on a dedicated thread, separate from the main algorithm.

3. Motion model with pre-computed geodesic distances

The proposed motion model consists of two steps: (a) For the given particle, the geodesic distances to all other faces are retrieved from a pre-computed lookup table. (b) Weights are assigned based on the geodesic distances, and the particle for the current time step is sampled from the resulting distribution. This approximates Gaussian sampling directly on the robot surface, requiring approximately 0.6 µs to update a single particle.

4. Flow-guided particle set management

For robust contact event detection, the conventional QP-based criterion is augmented by integrating generative estimates from the dual-proposal distribution. Increasing condition: satisfied only when the QP condition is met and the dual particles are widely spread, i.e., when their standard deviation within a set exceeds that of the motion model. Decreasing condition: a particle set is a candidate for removal if it is not assigned any dual particles during sample allocation, and it is removed if the QP-based condition supports this removal.

Results

Contact event detection (Experiment B)

Experiment B: Sequential snapshots comparing the baseline and proposed methods. (a) The baseline method suffers from erroneous set initialization due to local minima caused by particle starvation. (b) The proposed method effectively performs contact event detection without such false positives.

Real-world experiment (Experiment D)

For dual-sampling, the flow matching model trained in simulation was directly transferred to the physical robot. A dedicated evaluation dataset was collected using a specialized tool to acquire ground-truth contact locations and forces (a 6-axis F/T sensor for force tracking and OptiTrack markers for pose tracking). A total of 200 trials were conducted across 20 distinct robot configurations.

Additional real-world scenarios

Full video

BibTeX

@article{han2026mixture,
  author={Han, Seo Wook and Kim, Min Jun},
  journal={IEEE Transactions on Robotics},
  title={Mixture of Flow Matching and Particle Filter for Proprioception-Based Online Contact Estimation},
  year={2026},
  pages={1--20},
  doi={10.1109/TRO.2026.3723975}
}