Hello author, I'm not quite familiar with NeRF volume rendering. Could you explain why probability accumulation is summed up here to render depth? What is the corresponding mathematical formula for this process? What is the corresponding physical meaning? in my opinion,cumulative multiplication might make more sense?
def get_density(self, rays_o, rays_d, Voxel_feat, is_train, inputs):
dtype = torch.float16 if self.opt.use_fp16 else torch.float32
device = rays_o.device
rays_o, rays_d, Voxel_feat = rays_o.to(dtype), rays_d.to(dtype), Voxel_feat.to(dtype)
reg_loss = {}
eps_time = time.time()
with torch.no_grad():
rays_o_i = rays_o[0, ...].flatten(0, 2) # HXWX3
rays_d_i = rays_d[0, ...].flatten(0, 2) # HXWX3
rays_pts, mask_outbbox, z_vals, rays_pts_depth = self.sample_ray(rays_o_i, rays_d_i, is_train=is_train)
dists = rays_pts_depth[..., 1:] - rays_pts_depth[..., :-1] # [num pixels, num points - 1]
dists = torch.cat([dists, 1e4 * torch.ones_like(dists[..., :1])], dim=-1) # [num pixels, num points]
sample_ret = self.grid_sampler(rays_pts, Voxel_feat, avail_mask=~mask_outbbox)
if self.use_semantic:
if self.opt.semantic_sample_ratio < 1.0:
geo_feats, mask, semantic, mask_sem, group_num, group_size = sample_ret
else:
geo_feats, mask, semantic = sample_ret
else:
geo_feats, mask = sample_ret
if self.opt.render_type == 'prob':
weights = torch.zeros_like(rays_pts[..., 0])
weights[:, -1] = 1
geo_feats = torch.sigmoid(geo_feats)
if self.opt.last_free:
geo_feats = 1.0 - geo_feats # the last channel is the probability of being free
weights[mask] = geo_feats
# accumulate
weights = weights.cumsum(dim=1).clamp(max=1)
alphainv_fin = weights[..., -1]
weights = weights.diff(dim=1, prepend=torch.zeros((rays_pts.shape[:1])).unsqueeze(1).to(device=device, dtype=dtype))
depth = (weights * z_vals).sum(-1)
rgb_marched = 0```
Hello author, I'm not quite familiar with NeRF volume rendering. Could you explain why probability accumulation is summed up here to render depth? What is the corresponding mathematical formula for this process? What is the corresponding physical meaning? in my opinion,cumulative multiplication might make more sense?