[Work Log] Debugging reconstruction anomalies
Continuing from yesterday. Issue: strange reconstruction results after retraining.
Issue: Reconstruction anomalies
Experiment #1
Add a small amount to the diagonal of the covariance and re-run reconstruction.
Result: Adding a moderate amount of covariance to the diagonal seems to fix the results but probably due to additional smoothing.
Adding 1.5 - makes results worse, bizarre jutting sections. This counterintuitive if the phenomenon is due to increasing noise variance. Adding 2 - makes reconstruction look good. Adding 10000 - reconstruction becomes straight sticks.
Discussion: analytically this is equivalent to multiplying the noise variance by a constant. We already know that increasing noise variance solves the issue. The question remains: why did our training algorithm prefer this? Also, why does decreasing noise variance force the reconstruction away from the data? Strong contradiction between data and
Experiment #2
Add a small amount to the diagonal of the prior covariance matrix
Result: no obvious improvement (inconclusive)
Experiment #4
Disable attachment
Result: significant improvement.
Discussion: This was totally unexpected. Almost all issues seem to be caused by bad attachment. The affect of bad attachment is probably exacerbated by small noise variance -- the hardness of the constraints lead to contradictions that are resolved by anomalous reconstructions.
Conclusion
Bad attachment guess, combined with nonrobust choice of nosie variance causes contradiction in posterior and strange reconstructions. Need to develop a better approach to infer attachment. Issue there is partly that reconstructed individual curves have tails that extend past the attachment point, likely due to bad index estimatization. If we fix index estimation, we'll be closer to a good branch estimation procedure.
Misstep - index optimization "bug"
Noticed camera linearization was occuring after index optimization, which is likely causing the looping.
Added optional camera linearization to process_tracks.m. Issues seemed to get worse, but in fact, linearization wasn't running -- forget to set flag to 'true'.
Overextension
Experiment #1
Hypothesis: overrestrictive noise variance results in contradictions requiring contortion to resolve.
Approach: relax noise
Results:
- 2x - much worse
- 4x - different, still bad
- 8x - still bad
- 16x - mostly resolved; weak tails
Is linearization causing it? is model 5 causing it? Is linearization trsutworthy?
Linearization seems to make it a little worse.
Should we take linearization into account when optimizing depth?
scenario: curve points toward camera, high uncertainty as to index, because BP line is in direction of curve. After linearization, BP lines change; no longer in direction of curve, index optimization can't fix bad 3d position.
Miscellaneous thoughts
It kind of makes sense that iid noise would be zero, since the data we're drawing from is so smooth to begin with, and basically noiseless. Any errors arise from mistracing and are strongly correlated due to the smoothness of the Bezier curves.
If the issue is near-singular matrices, the traditional solution to this is adding a tiny amount of covariance to the diagonal of the prior matrix.
Open issues
Index optimization resulting in weird loops at endpoints
Camera linearization reuslts in extensive stretching and looping during index optimization
T
Posted by Kyle Simek