BEDA

BEDA is a visualization and analysis tool that helps behavioral scientists to easily incorporate sensor data analysis into their behavioral data analysis procedures. Behavioral scientists often collect video recordings of human behavior across multiple sessions to understand how therapy affects the behavior of their subjects. Additionally, as technology advances, researchers have begun collecting various types of physiological data (e.g., temperature, heart rate, skin conductance level) that can help them better understand individuals' behaviors. However, because most of behavioral scientists are not familiar with physiological data analysis, we developed BEDA to assist their data analysis.

Computer science researchers collaborated with behavioral scientists in a research study collecting multiple sessions of video, sensor, and human-coded behavior data. Our aim was to understand the difficulties behavioral scientists encounter when analyzing the collected data.
The purpose of the behavioral science research study was to examine if changes in a child's level of engagement with a task and/or the level of undesired behaviors were associated with changes in electrodermal activities (EDA) that measures change in electrical skin conductance.

Managing more than 100 files using more than 4 software programs

Difficulties in analyzing sensor data and interpreting the analysis results

Disconnection between the behavioral and physiological analyses introduced opportunity for error in interpreting these results.

For example, to identify how EDA data changed when undesired behaviors occured, researchers needed to individually note the time points when undesired behaviors occurred in the behavioral analysis in the Word document (e.g., undesired behavior occurred at 1 minute, 10 seconds) and then examine the corresponding time point of the sensor signal in the sensor signal visualization. This comparison process had to continue for each occurrence of an undesired behavior within each session across the 26 intervention sessions.

To mitigate these difficulties, we created a tool, Behavioral and External Data Analysis (BEDA).

Presenting multiple sessions of video and physiological sensor data streams concurrently

The researchers inspected both behavioral coding and sensor signals to identify patterns or anomalies between the data across the sessions. The changes in EDA typically correlated with occurrences of undesired behaviors for this child. For example, in the first session, there is a steep increase in the phasic EDA level after self-injurious and stereotypic behaviors started to occur. By looking at the third and fourth sessions, researchers found this pattern continued even when receiving pressure from the vest.

Supporting analysis procedures for sensor and behavior data

For researchers who are not familiar with the sensor data analysis procedures, BEDA embeds sensor data analysis algorithms and provides an interface (a) that allows them to simply select and apply an algorithm on their data.

Visually annotating results within the original data signal

Visualizations of sensor data analysis results that are applied as users run multiple data analysis scripts (c). The first visualization is the raw sensor data. The next visualization is the result after running smoothing, separating tonic and phasic, and the "count peaks" analysis scripts on top of the raw data. Tonic signal is visualized on top of the raw sensor data and the remaining signal on top of tonic signal represents phasic signal. The third visualization is the results when the parameters of the "count peaks" script were adjusted (b).

We successfully piloted this tool with behavioral scientists and found that BEDA facilitated data management and analysis. An unanticipated benefit of BEDA was that it allowed behavioral scientists to evaluate categories of behaviors that they defined for analysis and increased their confidence in conducting analyses of sensor data.

Saved time when organizing and aligning multiple data sources

"Instead of having twenty Microsoft Word files containing the behavioral data for each session, the twenty video files representing each session, and the twenty sensor data files representing each session, BEDA allowed us to have a single file containing all of the data for the study aligned and synchronized. This reduced both the confusion and amount of time required for managing the data."

Raised confidence to conduct sensor data analysis

"I don't know anything about signal processing and how to use R or MATLAB to write code to process signals. With BEDA, I can do sensor data analysis on my own, see how this analysis altered the sensor data, and understand them while looking at the behavioral coding and identify things that are happening. It makes multimodal data analysis doable for me as a behavioral scientist who has no skills or training in sensor signal processing or analysis."

Validated behavior coding and facilitated secondary analysis of behavior

For example, the behavioral scientists had defined screaming and hitting himself as undesired behaviors for a child participant that they hypothesized to be correlated with high arousal level. However, they did not know for sure how these behaviors would actually affect his internal status. Using BEDA, with sensor data and behavioral coding visualized across multiple sessions, the research team member who participated in the interview evaluated if the behavioral coding the team had defined was actually related to EDA data changes.

"Understanding underlying causes for behavior and validating some of the behavioral coding that we already have"