[Work Log] Debugging WACV errors
Troubleshooting reconstruction anomalies.
- significant translational offset
- Why is reconstructed base point moving? Initial points should be 100% corellated.
Observations during debugging:
- Observation indices start at negative values
- Initial point in view 2 is 5mm past initial point in view 9
- translational offset was due to auto-centering feature of
view_all_wacv.m. Changed default to "no auto centering" - Solved: Base point movement was due to auto-centering.
Okay, now on to bigger issues: why is tip moving so much?
Theory: converging center-lines without converging uncertainty causes drift toward camera and is untenable.
Could re-shaping the uncertainty cylinders help?
Also strange: perturb variance is huge - probably to allow curves to shift toward the camera, exploiting the "singular region" phenomenon studied earlier.
Singular regions
These are becoming the main struggling point. It is likely that the endpoint drift is caused by this too. We could use fully linear cameras, which would avoid converging of uncertainty cones, but we lose some of the novelty of point-wise linearization.
Had the idea of iteratively reconstructing and re-estimating the uncertainty cylinders, but this is not guaranteed to be monotonic -- in fact it seems likely to reduce the solution's true likelihood at each iteration. First it would drift toward the camera; then we shrink the covariance cylinders, which reduces the likelihood; repeat.
Perhaps the reason point-wise linearization is "novel" is because everyone knows it doesn't work, but no one has published on that fact. Or I haven't reviewed that early literature enough.
Maybe PWL (point-wise linearization) is bad for reconstruction but still okay for marginalization? Or maybe it's just causing our learning to fail, but once we learn reasonable parameters, the pathological behavior disappears? Assuming reasonable values on perturbation variance, we shouldn't see significant drift toward cameras (how much?).
Full-camera Linearization
Should try full-camera linearization. Can we implement this quickly without changing much infrastructure? Probably. Find the mean direction and replace all data covariances with the one in that direction. New matrix -- First eigenvector: mean direction; second and third: doesn't matter (Gram-Schmidt from old eigenvectors?). If original second and third eigenvalues aren't equal, use their mean for new ones.
This is kind of a hybrid of point-wise linearization and full-camera linearization, because the global direction is camera-wise, but each point's variance is different. Equivalently, this is full-camera linearization with nonuniform point weights (far points have lower weight, higher variance).
As long as we're abandoning some novelty, could go with traditional full-camera linearization, which might yield a more efficient index optimization scheme? Should compare the different approaches.
Diagonal data covaraince means we can use the technique from this 2011 NIPS paper.
TODO
- Implement
- re-train, re-run reconstruction, check for