Add Neuropixels saturation levels estimation from probe table/metadata

[alejoe91]

This PR estimates ap/lf gain and saturation levels from the ProbeTable and metadata.

For NP2.x and up, this is done directly in the build_neuropixels_probe, since the gain is fixed and in the ProbeTable.

For NP1.x, the ap and lf gains are retrieved from the imro header/table for SpikeGLX and from the settings file in OpenEphys.

[h-mayorquin]

Is this related to the neo PR? I will take a look tomorrow.

[alejoe91]

Is this related to the neo PR? I will take a look tomorrow.

Not really, it's more related to this: SpikeInterface/spikeinterface#4539

Basically, from the probetable and metadata files we can figure out the saturation levels for Neuropixels probes. This will allow to automatically set saturation thresholds for the new SI function to detect and remove artifacts!

[h-mayorquin]

I agree with the goal of making saturation easy for downstream consumers to compute. I have a request about how the information is split across the catalogue and recording layers.

I would like to keep separating probe catalogue information (what build_neuropixels_probe does) from experimental setup configuration (what read_spikeglx and the other format readers do) (see issue #405). From that perspective, adding adc_range_vpp as a probe-level annotation in section 7 of build_neuropixels_probe is exactly right. What I like less is conditionally writing the gains at the catalogue level too. The catalogue can only do this for NP2.x where the gain is hardware-fixed, so the annotation ends up present on NP2.x probes and absent on NP1.x ones, and consumers have to know which family they are looking at.

What I would suggest is exposing the full gain spec at the catalogue level and building the per-contact values at the recording level. Concretely: have build_neuropixels_probe annotate adc_range_vpp, ap_gain_list, and lf_gain_list (the latter two are at most eight elements). Then have read_spikeglx and read_openephys build ap_gains and lf_gains as per-contact arrays, populated from the IMRO table or settings.xml for NP1.x and from the catalogue spec list for NP2.x. With those in place, saturation becomes easy to calculate, any consumer derives it inline:

adc_range_vpp = probe.annotations["adc_range_vpp"]
ap_gains = probe.contact_annotations["ap_gains"]
ap_saturation_uV = (adc_range_vpp / 2.0) / ap_gains * 1e6 

This can be done in SpikeInterface (if we care about reducing property memory) or here if we prioritise convenience. Both choices make sense to me but I have a slight preference for doing it on SpikeInterface so we don't commit to the uV instead of adc units here.

As side benefits, this will also make the code easier to read by reducing the ifs and the asymmetries between versions (see cyclomatic complexity), and it will be useful for python-neo PR #1842, because the per-contact gain shape matches what neo already produces as channel_gains, so neo could eventually consume these annotations directly instead of recomputing gain from imAiRangeMax in its own SpikeGLX reader.

What do you think?

[h-mayorquin]

Could we check #417 before this as it came before so we don't deviate that much.