Feature extraction includes occluded views without visibility checking

[BIG-PIE-MILK-COW]

Description

The current implementation in extract_feature.py extracts DINOv2 features from all views and averages them without checking if a 3D point is actually visible in each view. This leads to feature contamination when points are occluded or on back-facing surfaces. Won't this have a significant impact on the VAE's performance? Or are there other processing details I haven't noticed? Looking forward to your response, thank you.

[ZhexiLuo]

We are encountering the same issue. Looking at extract_feature.py, the current projection logic has a visibility bug:

When a voxel projects outside the camera frustum (uv outside [-1, 1]), grid_sample returns zero due to zero-padding — but the feature accumulation still divides by n_views (total view count). This incorrectly dilutes the aggregated feature magnitude for voxels only partially visible across views.

Proposed fix:

1. Depth check: use z from project_cv()[1] to reject back-projected points (z <= 0)
2. UV bounds check: mask out samples where |uv| > 1
3. Occlusion check: compare projected depth against rendered depth map for self-occlusion
4. Per-voxel normalization: divide by actual visible view count per voxel, not fixed n_views

Something like:

valid = (z > 0) & (uv.abs().max(dim=-1).values <= 1)
# optionally: valid &= (pts_depth <= gt_depth + threshold)
feat_sum[valid] += sampled[valid]
count[valid] += 1
feat = feat_sum / count.clamp(min=1)

Is this intentional (soft-weighting design), or a known limitation? Happy to contribute a fix.