except in light of behavioral annotations

Behavior is complex and complicated

• Stimulus events often have relationships which behavior & neural activity depend on (e.g. Match to Sample tasks)
• Behavioral events may be inter-related through hierarchical structures (e.g. birdsong)

Operant conditioning as a case study

Stimulus ❱ Response ❱ Consequence

• Which event was the Response? Stimulus? Consequence?
• Was the trial correct?

Basic Operant Conditioning

						Time         | ID | Stimulus | Response | r_Time | Error
-------------|----|----------|----------|--------|------
12:13:05.265 | 1  | puppy    | Right    | 2.115  | False
12:13:14.655 | 2  | kitty    | Right    | 2.004  | True
12:13:87.098 | 3  | puppy    | Left     | 2.298  | True
						
					

This is a typical data format when only doing a simple behavior

Match to Sample

						Time         | ID | Sample | Target | Response | r_Time | Error
-------------|----|------- |--------|----------|--------|------
12:13:05.265 | 1  | puppy  | puppy  | Right    | 2.115  | False
12:13:14.655 | 2  | kitty  | puppy  | Right    | 2.004  | True
12:13:87.098 | 3  | puppy  | kitty  | Left     | 2.298  | False
						
					

However, if we are doing neurophysiology, we need to know timing information

Need timing of Sample & Target

						Time         | ID | Sample | s_Time | s_Dur | Target | t_Time | t_Dur | Response | r_Time | Error
-------------|----|------- |--------|-------|--------|--------|-------|----------|--------|------
12:13:05.265 | 1  | puppy  | 0.0    | 0.5   | puppy  | 1.5    | 0.5   | Right    | 2.115  | False
12:13:14.655 | 2  | kitty  | 0.0    | 0.5   | puppy  | 1.0    | 0.5   | Right    | 2.004  | True
12:13:87.098 | 3  | puppy  | 0.0    | 0.5   | kitty  | 1.2    | 0.5   | Left     | 2.298  | False
						
					

However..

• Sample, Target & Response are each unique Events
• The Class of a trial is determined by the Sample/Target relationship

Solution: Grouping stimulus, response, and consequence events into Trials

						Events
======
Time         | ID | Class  | Value  | Duration
-------------|----|--------|--------|---------
12:13:05.265 | 1  | sample | puppy  | 0.5
12:13:06.765 | 2  | target | puppy  | 0.5
12:13:07.380 | 3  | resp   | left   | --
12:13:07.387 | 4  | conseq | reward | 1.0       
12:13:14.655 | 5  | sample | kitty  | 0.5
12:13:15.655 | 6  | target | puppy  | 0.5
12:13:16.659 | 7  | resp   | left   | --
12:13:16.668 | 8  | conseq | punish | 5.0    

Trials
======
ID | Sample_id | Target_id | Resp_id | Conseq_id | Task 
---|-----------|-----------|---------|-----------|------
1  | 1         | 2         | 3       | 4         | match
2  | 5         | 6         | 7       | 8         | match
						
					

...or..

Defining Target-Sample relationship with references between events

						Time         | ID | Class  | Value  | Duration | Sample_id
-------------|----|--------|--------|----------|----------
12:13:05.265 | 1  | sample | puppy  | 0.5      | --
12:13:06.765 | 2  | target | puppy  | 0.5      | 1
12:13:07.380 | 3  | resp   | left   | --       | --
12:13:07.387 | 4  | conseq | reward | 1.0      | --
12:13:14.655 | 5  | sample | kitty  | 0.5      | --
12:13:15.655 | 6  | target | puppy  | 0.5      | 5
12:13:16.659 | 7  | resp   | left   | --       | --
12:13:16.668 | 8  | conseq | punish | 5.0      | --
...
						
					

My point here is not to argue for a standard for representing behavior data

Rather, that for even a relatively simple & common behavioral design, relationships between events are important

More complex relationships underly more complex behaviors

• Task switching on cue
• Hierarchies in motor control
• Affect of recent outcomes on decision biases

What does this mean for NeuroData?

Allow for Normalization of Behavioral Events

• SQL — Foreign keys
• HDF5 — reference datatype
• NoSQL — References or links

This opens up behavioral representations to be a rich as the behaviors themselves

and lets the lines between behavior and neurophysiology be drawn as the researcher sees fit

Thank you

Justin Kiggins
Doctoral Candidate in Neurosciences, UC San Diego
Gentner Lab
jkiggins@ucsd.edu

http://neuromusic.github.io/talks/NWB-2014