[Work Log] Testing likelihood #2 (2-day)
Client send/receive timeout
Tried to get send/receive to timeout if the server didn't respond. It turns out, although this feature exists in native Unix sockets, the boost library abstractions render them useless. To get this, I'll need to use asynchronous IO, which I'm not ready to jump to, yet (also not sure if callbacks will work in mex, since the callback code might not exist after the constructor function returns).
Let's just take the approach that the server must always respond quickly or disconnect.
Testing curve_tree_ml_2
Need a reliable way to get a testing Trackset. Write something for wacv -- get_wacv_trackset.
Modified run_all_wacv to save Tracks; re-running on all datasets. get_wacv_results now returns Tracks as well as means.
Tweaked semantics so this now only computes the pixel likelihood. The full likelihood is the sum of this and curve_tree_ml.
Tweaked construct_attachment_covariance_2.m so the self-covariance matrices are computed in the function (if needed) instead of being passed in.
Spin-off construct_attachment_covariance_3.m a fully general version that receives input and output indices, and optionally a pre-computed self-covariance.
TODO: make this the "official"/dispatch version, make other versions call this (or eliminate altogether)
Performance
data matrix is HUGE. 8913 dimensions. Is this right? Also, we can move inversion outside of the loop (maybe).
Consider 'subset of data' method, or other dimensionality-reduction method
Issue: Can't do cholesky decomposition, because points branching from the base are redundant. Using SVD instead.
Okay, it appears to be successfully running end-to-end. Haven't confirmed results yet, but one thing is clear... its REALLLLLY slow (3 minutes and counting)
Can do ancenstral sampling to exploit tree structure to speed it up. can further use markov property to break up curves.
Oops, not quite end-to-end success. Some indexing, reshaping issues. Other bugs found
- wasn't adding posterior mean to sampled results
- missing transpose in posterior covariance equation.
Debugging sampled curves
Message looks okay under inspection, but getting -inf (due to exception). Dumping...
Found a recent bug in code that builds the Gl_curve. When I chnaged an assert to an if/throw, I forgot to negate the conditional.
Getting finite values now.
Result looks in the ballpark, but some perturbations look questionable (considering the tight constraints on the WACV dataset).
Doind a full dump-mode run...
Found a bug in a recent refactor of dump-mode, causing segfaults.
All views drift very far from their true values. Possibly a math bug in the magnitude of the posterior covariance?
Refactored server's "dump mode" to continuously dump each message as it's received, instead of running in offline mode and dumping only the passed-in model.
Trying method2. Got similar results, qualitatively; per-view curves still have some bizzarre features. Interestingly, the perturbations between successive samples of method 2 (and between method2 and method 1) are relatively small, suggesting this is an issue with the mean, not the variance.
Recall that we only tested the no-perturb model for wacv reconstruction; not the per-view reconstruction.
Re-running WACV dataset 2 with OU-perturb model.
Looks sensible. So that rules out the parameter settings causing bad mu's. We should be getting exactly WACV results; can we get there?
- use K_star = K
- use zero covariance matrix (always sample at the mean)
Running... (slow, because matrix multiplication is so much larger)
I'm guessing the bug is in the full-tree covariance .
...
Getting "degenerate curve" error. Thinning points fixed.
Dumped results look good. This suggests that mean math is likely correct, as long as K_star is okay. So K_star math is probably wrong.
Need to write a test to confirm and start debugging.
...
Done. Results match on symmetric and non-symmetric index sets. Which means no progress made on this bug...
...
Re-ran, but using non-zero Sigma. Resulting covariance was not positive definite; SVD was complex-valued.
Possibly this is a result of the same bug?
Covariance matrix Rank-reduction
While previous test was running, read-up on reduced-rank approximations to K. Nystrom method seems sensible to speed up matrix inversion. Doesn't avoid the matrix multiplication with K_star, or cholesky decomposition of the posterior covariance... but using a smaller output index set seems like a reasonable approach to mitigating both of those.
Curious how many non-negligible eigenvalues we have; how many data points we should use. Recalling a plot of eignevalues from a few days ago, it looked like less than 0.1% of the dimensions are significant, but need to get a concrete number.
Alternative approach is to use the problem's unique markov structure to sample each section piecewise. Needs some thought for how to do in the case of the posterior; probably need to estimate a markov blanket for each, along with sampling each branch point and conditioning on it. Upside: better chance at asymtotic running time improvement. Down-side: not general, not as re-usable within the project.
To Be Continued
Bug: Samples generated from curve_tree_ml_2_debug.m are wonky.
Theory: Bug is related to posterior covariance. Posterior mean looks sensible.
File: curve_tree_ml_2_debug.m <-- alternative version, where K is used for K_star and K_star_star. Currently giving non-positive-definite results for posterior covariance using method 1. Currently Stumped
Fix sampling sigma bug.
TODO (new)
- get likelihood from multiple samples -- what is the variance of the MC estimator?
- re-run wacv with multi-view data, but no-perturb output