Useful functions to reshape/arrange/reduce raw data into clean data to add to the record

Synchronisation utils

extend_sync_timepoints[source]

extend_sync_timepoints(timepoints:ndarray, signals:ndarray, up_bound, low_bound=0)

Extend arrays of timepoints and signals (with identical shape) from the low_bound up to the up_bound. For example, the first timepoint could be 2000, and with a low_bound of 0, it would add the timepoints 0, 500, 1000, 1500 if the timepoint distance is of 500 (obtained by averaging the timepoints distances).

params:

- timepoints: Timepoints to extend
- signals: Signals to extend
- up_bound: Up bound to which to extend both timepoints and signals
- low_bound: Low bound to which to extend both timepoints and signals

returns:

- timepoint: Extended timepoints
- signals: The datachunk array is not modified, but the idx attribute is increased by the number
of frames added with the low_bound.

align_sync_timepoints[source]

align_sync_timepoints(timepoints:DataChunk, signals:DataChunk, ref_timepoints:DataChunk, ref_signals:DataChunk)

Align the signals of a timepoints timeserie to a reference ref_timepoints with the corresponding ref_signals. ref_timepoints is extended to match ref_timepoints lenght.

params:

- timepoints: timepoints to align
- signals: signals to align
- ref_timepoints: reference timepoints
- ref_signals: reference signals

return:

- Aligned timepoints (DataChunk)
- Aligned signals (DataChunk)

resample_to_timepoints[source]

resample_to_timepoints(timepoints:ndarray, data:ndarray, ref_timepoints:DataChunk, group='data')

Resample the data at timepoints to new timepoints given by ref_timepoints. Return a DataChunk of the resampled data belonging to a specified group.

params:

- timepoints: Original timepoints of the data
- data: Data to resample of shape (t, ...)
- ref_timepoints: Target timepoints for the resampling
- group: Group assigned to the returned DataChunk

return:

- Resampled datachunk with appropriate idx.

link_sync_timepoints(frame_tp_1, frame_sig_1, frame_tp_2, frame_sig_2)

Creates timepoints between two timepoints array sampled at the same rate. This is usefull for the LED dome which cannot generate frames in between stimuli (due to ROM update)

params:

- frame_tp_1: Timepoints of the first part
- frame_sig_1: Signals of the first part
- frame_tp_2: Timepoints of the second part
- frame_sig_2: Signals of the second part

return:

- (concatenated_frame_timepoints, concatenated_frame_signals)

flip_stimulus[source]

flip_stimulus(stim_inten, ud_inv, lr_inv)

Flip QDSpy stimuli arrays to match the up/down left/right orientation of the stimulus displayed to the mouse.

params:

- stim_inten: Stimulus matrix to flip of shape (t, color, y, x)
- ud_inv: Up down inversion boolean (1 to make the flip, 0 for no operation)
- lr_inv: Up down inversion boolean (1 to make the flip, 0 for no operation)

return:

- Flipped stimulus array

flip_gratings[source]

flip_gratings(stim_shader, ud_inv, lr_inv)

Flip gratings to match the up/down left/right orientation of the stimulus displayed to the mouse. A grating is encoded by an array of shape (t, 3(size, angle, speed)). Therefore the angles of the grating are modified to encode the "flipped" grating.

params:

- stim_shader: Grating matrix to flip of shape (t, 3(size, angle(degree), speed))
- ud_inv: Up down inversion boolean (1 to make the flip, 0 for no operation)
- lr_inv: Up down inversion boolean (1 to make the flip, 0 for no operation)

return:

- Flipped grating array

stim_to_dataChunk[source]

stim_to_dataChunk(stim_inten, stim_start_idx, reference:DataChunk)

Factory function for DataChunk of a stimulus, that squeeze the stim_inten matrix.

params:

- stim_inten: Stimulus matrix of shape (t, ...)
- stim_start_idx: Starting frame index of the stimulus
- reference: DataChunk signal reference used to determine the starting index of the stimulus

return:

- Datachunk of the stimulus

phy_results_dict[source]

phy_results_dict(phy_dir)

Open the result arrays of spike sorting after manual merging with phy.

params:

- phy_dir: path to the phy results

return:

- Dictionnary of the phy arrays (amplitudes, channel_map, channel_positions, spike_clusters,
spike_templates, spike_times, templates)

spike_to_dataChunk[source]

spike_to_dataChunk(spike_timepoints, ref_timepoints:DataChunk)

Factory function of a DataChunk for spiking count of cells from spike timepoints.

params:

- spike_timepoints: Dictionnary of the cells spike timepoints (list)
- ref_timepoints: Reference DataChunk to align the newly created spike count Datachunk

return:

- Spike count datachunk of shape (t, n_cell)

get_calcium_stack_lenghts[source]

get_calcium_stack_lenghts(folder)

Function to extract calcium stack lenghts from imageJ macro files associated to the stacks.

params:

- folder: path of the folder containing the IJ macros files

return:

- list of stack lenghts

twoP_dataChunks[source]

twoP_dataChunks(ref_timepoints:DataChunk, frame_timepoints, len_epochs, *args)

Factory function for two photon data.

params:

- ref_timepoints: Reference timepoints to create the DataChunk
- frame_timepoints: List of frame timepoints for each sequence of two photon frame recorded.
- len_epochs: Lenght of the recorded epochs (<= than the corresponding frame_timepoints). Int of list
- args: matrices of all frames detected by CaImAn. (give as many as you want to synchronise)

return:

- tuple containing the synchronised matrices in the order it was given

Modelling utils

img_2d_fit[source]

img_2d_fit(shape, param_d, f)

Helper function to generate the 2D image of a fit.

params:

- shape: Shape of the image in (y, x).
- param_d: Fit dictionnary.
- f: Function used of the fit.

fill_nan[source]

fill_nan(A)

Fill nan values with interpolation. Credits to BRYAN WOODS@StackOverflow

Processing utils

stim_inten_norm[source]

stim_inten_norm(stim_inten)

Normalize a stimulus with intensity in the 8bit range (0-255) to -1 to 1 range.

group_direction_response[source]

group_direction_response(stim_prop, spike_counts, n_repeat, n_cond=32)

Group the cells responses from shuffled grating stimulus repetitions. Retrieves a dictionnary with a key for each condition.

params:

- stim_prop: Grating array of shape (t, 3(size, angle, speed))
- spike_counts: Spike counts response of the cells of shape (t, n_cell)
- n_repeat: Number of repeat of each condition
- n_cond: Total number of condition (speed/size condition * n_angle)

return:

- dictionnary of the spike counts for each condition (speed/size), with shape (n_angle, n_repeat, len, n_cell)

group_chirp_bumps[source]

group_chirp_bumps(stim_inten, spike_counts, n_repeat)

Find the cells response to the OFF-ON-OFF initial parts of the chirps.

params:

- stim_inten: Stimulus intensity array
- spike_counts: Spike counts array of shape (t, n_cell)
- n_repeat: Number of repetitions of the chirp stimulus

return:

- Dictionnary of cells response to the different ON or OFF stimuli

get_repeat_corrected[source]

get_repeat_corrected(stim_inten, spike_counts, n_repeats=10)

Apply shifts (detected during synchro) to the chirp repetition.

params:

- stim_inten: Stimulus DataChunk (containing the shifts and frame replacements info)
- spike_counts: Spike count matrix of shape (t, n_cell)
- n_repeats: Number of repeats of the chirp

return:

- aligned cells response to stimulus, of shape (n_repeat, t, n_cell)
- Number of duplicated frame per repetition.

removeSlowDrift[source]

removeSlowDrift(traces, fps=60, window=80, percentile=8)

Remove slow drifts from behavioral temporal traces such as locomotion speed obtained from the treadmill signal or pupil size obtained from the eye_tracking signal, by extracting a specified percentile within moving window from the signal.

params:

- traces: Behavioral temporal traces obtained from reM
- fps: Sampling rate
- window: Moving temporal window in seconds
- percentile: Percentile to be extracted within moving window

return:

- Filtered temporal traces

time_shift_test_corr[source]

time_shift_test_corr(spike_counts, behav_signal, n_tests=500, seed=1)

Compute the null distribution of correlation between behavioral signal and spiking signal with a time shift test.

params:

- spike_counts: Array with spike counts for a specific neuron and data chunk from the reM
- behav_signal: Array with behavioral signal for a specific neuron and data chunk from the reM
- n_tests: number of used shifted signals to compute distribution
- seed: seed for numpy function random.randint

return:

- null_dist_corr: Null distribution of correlation values

cross_corr_with_lag[source]

cross_corr_with_lag(spike_counts, behav_signal, behav, conversion_factor_treadmill=None, removeslowdrift=True, fps=60, seconds=30)

Compute cross-correlation with lag between behavioral signal and spiking signal. Process signals, compute null distribution of the correlation with a time shift test and values . Return cross-correlation array, null-distribution array and values for plotting.

params:

- spike_counts: Array with spike counts for a specific neuron and data chunk from the reM
- behav_signal: Array with behavioral signal for a specific neuron and data chunk from the reM
- behav : String with name of behavioral signal to be analysed
- conversion_factor : The value to convert the treadmill signal into cm/s
- removeslowdrift: Boolean:
    False - doesn't remove slow drifts from the signal
    True - removes slow drifts by extracting a specified percentile within moving window from the signal
- fps: Sampling rate
- seconds: Window in seconds of the correlation lag

return:

- crosscorr: Cross-correlation with lag array between behavioral signal and spiking signal
- corr_peak: Cross-correlation value at peak synchrony between behavioral signal and spiking signal
- p_value_peak: P-value of the peak cross-correlation value
- offset_peak: Temporal offset of the peak synchrony between behavioral signal and spiking signal in seconds
- null_dist_corr: Null distribution of correlation values (output of 'utils.cross_corr_with_lag')

get_inception_generator[source]

get_inception_generator(imageset_folder, len_set=25, width=500, height=281)

Return a function to obtain inception loop images from their index.

params:

- imageset_folder: Path to the folder of the image sets
- len_set: Number of images concatenated per set
- width: image width

return:

- Function to obtain inception loop images from their index.

group_omitted_epochs[source]

group_omitted_epochs(stim_inten, spike_counts, n_fr_flash=4, n_fr_interflash=4, n_fr_isi=100)

Group the cells reponse to the different omitted stimulus epochs conditions (n_flashes)

params:

- stim_inten: The intensities of the omitted stimulus in shape (t)
- spike_counts: Spikes counts of the cells in shape (t, n_cell)
- n_fr_flash: Duration of a flash (ON flash during OFF baseline, OFF flash during ON baseline)
- n_fr_interflash: Number of frames between two flashes (during an epoch)
- n_fr_isi: Number of frames between two epochs

return:

- response_d_ON, response_d_OFF: Dictionnaries of the cells responses for different number of flashes repetions. Each contain an array of shape (n_cell, n_repeats, len_epoch+n_fr_isi).

Plotting utils

get_shank_channels[source]

get_shank_channels(channel_positions, shank_dist_th=80)

Group the channels of a Buzsaki32 silicone probe into their shanks from the channel position.

params:

- channel_positions: List of channel positions
- shank_dist_th: Distance between channels in X to rule if on same shank or not

return:

- array of grouped channel index of shape (n_shank(4), n_channel(8))

format_pval[source]

format_pval(pval, significant_figures=2)

Helper function to format pvalue into string.

stim_recap_df[source]

stim_recap_df(reM)

Extract stimuli parameters (originally from the Database) to put them into a dataframe that will be displayed in the recapitulation plot.

params:

- reM: RecordMaster to extract stimuli parameters from

return:

- dataframe with the stimuli important informations