Barcode Tutorial

Installation

The BARCODE software can be installed as an executable application on both Mac and Windows. To install the software, following the link here, and download the zip file corresponding to your operating system of choice. The source code can also be directly downloaded on any system running Python 3.12 or later. To do so, download the source code from the repository here.

BARCODE requires the following packages:

  • Gooey: Version ≥ 1.0.8.1

  • ImageIO: Version ≥ 2.34.1

  • Matplotlib: Version ≥ 3.8.4

  • ND2: Version ≥ 0.10.1

  • Numpy: Version ≥ 2.0.1

  • OpenCV: Version ≥ 4.9.0.80

  • PIMS: Version ≥ 0.6.1

  • PyYAML: Version ≥ 6.0.2

  • Scipy: Version ≥ 1.14.0

  • Scikit-Image: Version ≥ 0.24.0

BARCODE was developed on the package versions listed above, and may not work on previous versions of the listed software packages. The executable version of the app includes these packages. These packages can be installed individually or using the included source code file requirements.txt. To use the source file installation, use the command pip install -r requirements.txt to install the required packages, then navigate to the source file in the terminal and run the software using the command python main.py.

Data Preparation

If the data is not in the form of a .tif or a .nd2 file, follow this section to prepare the data for the BARCODE platform. Otherwise, move on to Section 3. The easiest method to convert the data to a .tif file is through ImageJ or FIJI. Once installed run the application and open the desired file. Then save as a .tif file types.

General Walk-through

Interface platform of BARCODE program

When first using the BARCODE application, it is recommended to start with a single channel of a single file. To do so, use the File Selection setting to select a file. Clicking Browse will allow you to search your file system for .tif and .nd2 files, while selecting the Folder Selection setting and Browse will allow you to select entire folders. Then, choose either the Parse All Channels setting to scan every video channel, or Choose Channel to scan a specific channel. Selecting a channel less than 0 will result in reverse indexing of channels (i.e, selecting -1 will run the program on the last channel of every video scanned, rather than the first).

Then in the general settings, the following features should be turned on: Binarization, Optical Flow, Intensity Distribution, Include Dim Files, Include Dim Channels, Verbose, Save Graphs, and Dataset Barcode.

The Binarization, Optical Flow, and Intensity Distribution parameters control which branches of the program are run on the data. BARCODE uses all three modules to provide 17 unique metrics to quantify the structure and dynamics of the system. Disabling any of these settings will result in a faster runtime.

The Include Dim Files and Include Dim Channels parameters ensure the data is processed even if the videos are classified as dim, which is defined as the code as videos where the minimum intensity is greater than 2/e times the mean intensity. Turning these settings on will enable the program to process these videos, but will flag them as having potentially unreliable data outputs due to the low contrast.

The Verbose parameter provides additional information while the data is being processed, including time step updates, total processing time, and if the input video was classified as dim.

Lastly, the Save Graphs, Save Reduced Data Structures, and Dataset Barcode parameters are additional outputs besides the CSV file of the output metrics that provide more information on each of the processes performed on the data and generate a visual ID for the dataset. The Save Reduced Data Structures parameter is used to save the binarized images, optical flow fields, and intensity distributions used for the BARCODE analysis in the form of CSV files. Saving these allows for further data analysis and verification.

After running the application on a file or directory, a settings.yaml file will be created in the file directory that saves the settings selected. This file can be selected in the Configuration YAML file parameters to use the settings for future datasets.

Adjusting Parameters

This Section of the Tutorial discusses how changing parameters effects the output BARCODE and why you may or may not want to adjust these parameters.

Animated gif of sample file used for data in this tutorial.

Adjusting the Binarization Settings

Changing the Binarization Threshold

If the binarization output metrics don’t agree with the qualitative analysis of the data, the first step is checking the Binarization Threshold. The Binarization Threshold changes the cutoff point of the Binarization as a proportion of (1 + the threshold times the mean). This should be increased if the data is largely empty and the binarization is picking up significant noise and should be increased if the data contains important information at low and high saturation. The threshold is shown visually below with the starting data being transformed into the following binary datasets based on their respective threshold values shown below. As the threshold decreases more noise can be seen while as it increases more features are lost. The best threshold value for the following examples would be 0.10 or 0.25.

(Above) Original image from a .tif file before binarization. (Below) A series of binarized images of the original, showing increasing binarization thresholds from -0.25 to 0.25.

Changing the Frame Step

Whether interested in increasing the speed of the BARCODE execution or increasing the detail, changing the Frame Step allows the customization of the speed and precision of the Binarization branch. Decreasing the Frame Step will have the largest improvement on fast or chaotic data.

Changing the Frame Start and Stop Percent

The frame start and end percent determine the portion of the video in which the change in the void and the island area is measured. By default, these metrics compare the average maximum of the area of material and space between the last 10 percent of the video with the average of the first 5 percent and calculates the percentage change. Changing these parameters will allow the shifting and scaling of the second portion to narrow in on smaller periods or to investigate behavior at earlier periods.

Adjusting the Optical Flow Settings

Changing the Frame step

Whether interested in increasing the speed of the Optical Flow Branch or increasing the detail, changing the Frame Step allows the customization of the speed and precision of the Optical Flow Branch. Decreasing the Frame Step will have the largest improvement on fast or chaotic data with bulk flow or high local velocities.

Changing the Window Size

The Window Size parameter defines the size of the region around each pixel that is used to compute the local motion within the optical flow algorithm. Larger window sizes result in less noise being captured by the algorithm, but will result in more blurry motion fields, while smaller window sizes are more capable of detecting smaller movement within the material, but will be more susceptible to random noise.

Changing the Downsampling

Depending on the scale of movement within the data, changing the downsampling rate can drastically increase the output of the Optical Flow measurements. The downsampling rate is the proportion the flow field is reduced to before measuring the flow between frames, and determines the rate of spatial averaging of the flow fields. This leads to less noisy data with reduced accuracy and precision, while better detecting bulk movement. Increasing this value is recommended for large-scale movement of areas of material. Decreasing is recommended for the movement of many smaller areas of material. The effect of the flow field by the downsampling rate is shown below.

Optical flow field visualizations at different downsampling rates, averaging 1, 8, and 16 pixels per vector.

Calibration parameters

The two remaining parameters in the Optical Flow settings are the Nanometer-to-Pixel ratio and the Frame Interval. Both of these can be changed to their corresponding values determined by recording settings. This changes the units of the Mean Speed and Speed Change metrics from pixels per frame to nanometers per second. For ND2 files, these parameters are automatically read using the file metadata.

Adjusting the Intensity Distribution Settings

Changing the First and Last Frames

The First and Last Frames determine the portion of the video that the intensity distribution is calculated over. By default, these metrics are selected to compare the first and last 10 percent of the data. Changing these parameters will change the start and endpoints of the analyzed video frame segments to track the differences in selected times of the data.

Changing the Percent of Frames Evaluated

Whether interested in increasing the speed of the intensity distribution branch or changing the size of the measurement period, changing the Percent of Frames Evaluated allows the user to either decrease the percentage of frames to increase the speed of the branch while decreasing the compared periods to narrow in on specific behavior or decrease the speed while increasing the precision of the metrics over a larger period.

Combining CSV files and Generating BARCODEs

After the creation of the summary CSV files through the BARCODE application, these files can be manipulated and utilized without recalculating each of the metrics. To do so, in Execution Settings, select the Combine CSV Files/Generate Barcodes setting. Then, in the BARCODE Generator and CSV Aggregator, select multiple summary CSV files and combine them into a larger file generated in the specified folder. In addition, if necessary, create a new BARCODE for any of the input CSV files. This can be used to generate an aggregate BARCODE file from multiple datasets for comparison, or to generate a visual representation of the BARCODE CSV data. Additionally, the Metric Sort parameter selects a BARCODE metric to sort the data on, organizing the data from least to greatest along one of the 17 BARCODE metrics.

After selecting these features, start the BARCODE program. Once finished, look in the directory of the selected data and a summary file, a BARCODE Matrix PNG and a folder containing the generated graphs and data structures should be found.

Reading BARCODEs

After generating the BARCODE for a dataset, how should it be interpreted? The rows of the BARCODE matrix represent the different files and channels analyzed, listed in the order of execution. For a single-channel analysis, only one row will be present. The order of the output metrics is shown in the diagram below. The color of each metric is determined by its scaled value relative to its respective limits and is then plotted using the "plasma" colormap. The exact values for each metric can be found in the summary.csv file.

Two examples of BARCODE output metrics with names and variables, and limits of parameters.

FAQs

Can I analyze files that aren’t .tif or .nd2?

No, files must be converted to .tif or .nd2 to analyze with the BARCODE application. Follow Section 2 to convert your files to the correct format.

Can I analyze a directory with both .tif and .nd2 files?

Yes, the program has no issue when executing folders with multiple acceptable file types.

How can I speed up the BARCODE execution?

The easiest way to speed up the BARCODE execution is to increase the Frame Step of the Binarization shown in section 4.1.2 or Optical flow shown in Section 4.2.1 or decrease the Percent of Frames Evaluated shown in Section 4.3.2

Links and Resources

Windows and Mac Download

Windows and Mac: Link

README and Source Code

README and Source Code: Link

ImageJ and FIJI Download

ImageJ:Link

FIJI:Link