[Work Log] 2 Days: debugging WACV reconstruction
Now that gradient is working, lets try using fminunc to optimize indices.
Getting nonsense results. Looking into curve data.
...
Inspecting the data, it's clear that our method of linearly sampling points along the bezier curve is resulting in very jagged curves. Example in dataset 8, curve 7:
view 9
view 4
It's not totally clear how best to resolve this. Ideally, we would sample at a finer grain, but this caused big slow-downs for the longer curves. Could use coarse-grain sampling for long curves, but some curves are long in some views and short in others, and nonuniform sampling breaks some assumptions we make in the library. Furthermore, associations are unknown at the time of samplingz
It's possible
It's possible the bad reconstruction we're seeing from this curve isn't due to bad correspondence, but a bad indexing estimation (a later stage of inference). We see that although the correspondence places c7v9 toward the end of the 3D curve, our re-indexing code places it more spread out, but unevenly: the first point has index 4, while the subsequent points have indieces [21, 23, 27, 27]. We usually prevent large amounts of index-skipping during re-indexing, but possibly the second-pass refinement is destroying this.
After double-checking, realized view #5 is the biggest problem child, not #7.
Interestingly, #5 has relatively reasonable looking correspondence:
But the reconstruction (both attached and free) is terrible:
ll_means has really bad points at the beginning and end of the curve. It looks like tails might be handled poorly in corr_to_likelihood_2.m
Attempting to run with "FIX_WORLD_T = false" in corr_to_likelihood. Early probes suggest this improves things; however, now getting a crash when calling construct_attachment_covariance.m. Getting NaN's from curve #8. I've observed that curve #8's indices start with two zeros. Maybe this is causing our covariance algorithm to choke?
...
Got it: when we don't correct endpoints (i.e. FIX_WORLD_T = false), endpoints can get repeated, which makes our "initial direction" computation fail.
- Find out why duplicate endpoints occur (doesn't triangulation make this improbable?)
- Handle duplicated points gracefully when computing start point.
Spun off ../correspondence/corr_to_likelihood_2.m. Added some plotting code to see data vs. smoothed reconstruction. Reconstruction is particularly ugly for curve #5.
At first glance, it looks like end points are poorly localized, and then are preserved through the smoothing pass. But the last 12 points which are poorly localized hive quite a few correspondences, according to the correspondence table.
With pre-smoothed reconstructrion in green:
Tried using GP smoothing (implemented in reconstruction/gp_smooth_curve.m) instead of matlab's csaps(), but I'm getting weird tails.
Must be a bug in how I'm computing the covariance (which I'm doing by hand, since the necessary fields aren't constructed at that stage of the pipeline). I've been over it a few times... time to sleep and look again in the morning.
Eventually, chicken/egg approach might be the solution: optimize points, optimize indices, repeat.
It's clear that smarter smoothing doesn't improve reconstruction of curve #5.
TODO
- plot projected reconstruction vs. data
- Terrible
- plot camera centers
is it all bad cameras? why do others reconstruct okay?
Test: incrementally reconstruct, adding one curve each time
Reverse mode: started with views 9 & 8, then added 7, 6, etc.. Goes to pot at curve 4, gets progressively worse through curve 1
The purpose of corr_to_likelihood is to correct position and indexing errors made during naive triangulation. Let's test if that's happinging by viewing post-correction results.
...
Dramatic improvement over pre-correction results. Still significant change after adding curve 2, and esp. after curve 1.
Last 2 points and first 4 (?) are problematic.
trying camera recalibration. wrote a quick 7-point calibration: cal_cam_dlt.m. Also wrote a camera visualization routine: visualization/draw_camera.m. The newly calibrated results are clearly different, but not obviously better, at least by inspection.
TODO:
- touch up cameras for views 3 and 7.
- back up old cameras, copy-in new cameras, re-run wacv dataset 8. Is curve 5 improved?
- Try chicken/egg approach,
* note that maximizing indices is easier than quasi-newton: just arc-length parameterize it. - Troubleshoot FIX_WORLD_T issue that's causing everything to break
- Need to adapt gp-smooth to handle perturbations