Skip to content

Latest commit

 

History

History
484 lines (418 loc) · 24 KB

README.md

File metadata and controls

484 lines (418 loc) · 24 KB
Raw

The Research Analytics Suite (RAS), developed by Lane within Gire Lab at the University of Washington, is a comprehensive, open-source platform written in Python for aggregating and analyzing scientific data from diverse sources. RAS is designed to be free and accessible, addressing financial and accessibility barriers in scientific research.

Please note: RAS is currently under active development and is not yet ready for public use. This repository is intended for demonstration purposes and to showcase the project's structure and features while it is being developed.

Key Features:

  • Data Management Engine (DME): Filters and aggregates large, complex datasets from multiple sources.
  • Analytics Suite: Includes tools for research data analysis, advanced statistics, machine learning algorithms, and data visualization.
  • Preloaded Functions: Ready-to-use functions for common analysis tasks.
  • Custom Functions: Allows users to create and implement custom analysis functions.
  • Future Integration: Designed for compatibility with tools like DeepLabCut.

RAS aims to foster a collaborative research community, enabling scientists and researchers to share their analytic workflows and contribute to a repository of shared knowledge, accelerating scientific discovery through open collaboration.

Table of Contents
  1. Welcome
  2. Getting Started & Prerequisites
  3. Installation (Anaconda or Pip)
  4. Running Research Analytics Suite
  5. Project Structure
  6. Usage
  7. Contributing
  8. License
  9. Contact

Research Analytics Suite

The Research Analytics Suite (RAS) is a cutting-edge, open-source platform developed in Python to address the diverse needs of scientific data analysis. RAS stands out by offering a comprehensive suite of tools for data aggregation, management, and analysis, derived from various input sources such as pixel-tracking technology, accelerometers, and analog voltage outputs.

RAS aims to democratize access to powerful data analysis tools traditionally dominated by commercial software. By eliminating financial barriers, RAS empowers researchers, educators, and industry professionals to conduct sophisticated analyses without the associated costs.

Overall Mission

  • The Research Analytics Suite aspires to cultivate a collaborative research community. It envisions a platform where scientists and researchers can share their analytic workflows, collaborate on projects, and contribute to a growing repository of shared knowledge and resources. This collaborative spirit aims to accelerate scientific discovery and innovation by leveraging the collective expertise of the global research community.

  • By providing a versatile and accessible toolset, RAS not only enhances the efficiency and effectiveness of data analysis but also fosters a culture of open collaboration and shared progress in the scientific community.

Key Features

Data Management Engine (DME)

  • A robust system for filtering and aggregating large, complex datasets from multiple sources. The DME ensures seamless integration and handling of diverse data types, facilitating comprehensive and efficient data analysis.

Analytics Suite

  • Offers an extensive array of tools for research data analysis, including advanced statistical methods, machine learning algorithms, and data visualization techniques. The analytics suite is designed to be both powerful and flexible, catering to the specific needs of each user.
  • Preloaded Functions: A library of ready-to-use functions for common analysis tasks, enabling users to quickly apply standard methods without extensive setup.
  • Custom / User-Defined Functions: Allows users to create and implement their own analysis functions, fostering innovation and customization in research workflows.

Future Integration

  • RAS is designed with future compatibility in mind, aiming to integrate seamlessly with other leading tools in the field, such as DeepLabCut, to expand its capabilities further.

Developed With

RAS is built using a variety of powerful tools and libraries to ensure robust functionality and performance, including (but not limited to):

Package Description
Python Python: The core programming language used for the development of RAS.
Dask Dask: A flexible parallel computing library for analytic computing.
PyTorch PyTorch: An open-source machine learning framework for deep learning.
TensorFlow TensorFlow: An end-to-end open-source platform for machine learning.
Distributed Distributed: A library for distributed computing with Python.
DearPyGui DearPyGui: An easy-to-use, high-performance GUI framework for Python.
DearPyGui-Async DearPyGui-Async: An extension for DearPyGui to support asynchronous operations.
Cachey Cachey: A caching library for managing the lifecycle of cached objects.
Matplotlib Matplotlib: A comprehensive library for creating static, animated, and interactive visualizations in Python.
NumPy NumPy: The fundamental package for scientific computing with Python.

(back to top)

Getting Started & Prerequisites

To get a local copy up and running, follow the following steps.

Python (3.11)

  • You can check your Python version by running the following command in the terminal: 
    python --version
  • If not, you can download it here or use Anaconda.

Clone the Repository

git clone https://github.com/lane-neuro/research-analytics-suite.git

Navigate to the project directory

cd research-analytics-suite

(back to top)

Installation

Windows

Anaconda

  1. Create a new environment using the supplied environment_win_linux.yml file: 
    conda env create -f environment_win_linux.yml
  2. Activate the environment: 
     conda activate research-analytics-suite

    Continue to Running the Project

Pip

  1. Install the required packages using the following command: 
    pip install -r requirements.txt

    Continue to Running the Project

    (back to top)

Linux

[Prerequisite] PortAudio

  1. Install PortAudio using the following command in the terminal: 
    sudo apt-get install -y portaudio19-dev

Anaconda

  1. Create a new environment using the supplied environment_win_linux.yml file: 
    conda env create -f environment_win_linux.yml
  2. Activate the environment: 
     conda activate research-analytics-suite

    Continue to Running the Project

Pip

  1. Install the required packages using the following command: 
    pip install -r requirements.txt

    Continue to Running the Project

    (back to top)

MacOS

[Prerequisite] Homebrew

  1. You can install Homebrew using one [1] of the following methods:

    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

  2. Once Homebrew is installed, install the required packages using the following commands:

    brew install portaudio
    brew install hdf5

  3. Install RAS using Anaconda OR Pip (below).

Anaconda

  1. Create a new environment using the supplied environment_osx.yml file: 
    conda env create -f environment_osx.yml
  2. Activate the environment: 
    conda activate research-analytics-suite

    Continue to Running the Project

Pip

  1. Install the required packages using the following command: 
     pip install -r requirements.txt

    Continue to Running the Project

    (back to top)

Running Research Analytics Suite

Launch the Research Analytics Suite (RAS) using the following command in the terminal:

python ResearchAnalyticsSuite.py
  • This will launch RAS with the default settings.

See Command Line Arguments for launch argument customization

Command Line Arguments

You can provide the following command line arguments to customize the behavior of the Research Analytics Suite:

  • -g, --gui: Launches the Research Analytics Suite GUI
    • default: 'true').
  • -o, --open_workspace: Opens or creates a workspace at the specified directory.
    • default: '~/Research-Analytics-Suite/workspaces/default_workspace'

See the Examples section below for implementation examples.

(back to top)

Examples

Creating / Opening a RAS Workspace

You can specify the workspace to open or create using the -o or --open_workspace argument.

python ResearchAnalyticsSuite.py -o ~/Research-Analytics-Suite/workspaces/a_new_workspace
  • This will create a new workspace at ~/Research-Analytics-Suite/workspaces/a_new_workspace if it does not already exist.
  • If it does exist, it will open the existing workspace at that location.
  • Note: The path must be a directory, not a file.
    • So if you want to open a 'config.json' file located at ~/Research-Analytics-Suite/workspaces/look_another_workspace/config.json, you would instead use the following command: 
      python ResearchAnalyticsSuite.py -o ~/Research-Analytics-Suite/workspaces/look_another_workspace
    • This will open the workspace configuration file located within the ~/Research-Analytics-Suite/workspaces/look_another_workspace directory.

Launching RAS with the GUI

The GUI is the default mode of operation for RAS. However, you can explicitly specify the GUI mode by using the following command:

python ResearchAnalyticsSuite.py -g true
  • This will launch RAS with the GUI; this essentially has the same effect as running RAS without any command-line arguments.
  • This is the default mode of operation for RAS, providing an interactive interface for data analysis and visualization.

Launching RAS in Command-Line / Headless Mode

The GUI is the default mode of operation for RAS. However, you can run RAS in command-line mode only by using the following command:

python ResearchAnalyticsSuite.py -g false
  • This will launch RAS without the GUI, running in command-line mode only.

(back to top)

Project Structure

Fluid and subject to change as the project is developed further. Refer to the code for the most up-to-date information.

Operation Manager

The operation_manager package orchestrates and manages data processing operations within RAS.

  • chains: Handles sequences of operations.
  • control: Manages control mechanisms.
  • execution: Handles operation execution.
  • lifecycle: Manages the lifecycle of operations.
  • management: Includes operation management functionalities.
  • nodes: Manages operation nodes.
  • operations:
    • computation: Includes computational operations.
    • core: Provides core functionalities for operations.
      • inheritance: Provides functionalities for managing child operations in a parent operation.
      • control: Provides control functionalities for operations including start, pause, resume, and stop.
      • execution: Provides execution functionalities for operations including preparation and execution of actions.
      • progress: Provides functionalities for tracking and updating the progress of operations.
      • workspace: Provides functionalities for workspace interactions, loading, and saving operations.
    • system: Includes common system operations, such as ResourceMonitorOperation and ConsoleOperation
  • task: Manages all tasks associated with operations.

GUI

The gui package provides graphical user interfaces for interacting with RAS.

Note: The GUI package will be optional in future distributions, given RAS has real-time command-line interface integration.

  • assets: Contains GUI assets, such as images and icons.
  • base: Base GUI components.
  • dialogs: Contains dialog components, divided into subcategories:
    • data_handling: Dialogs related to data handling.
    • visualization: Dialogs for data visualization.
    • settings: Settings-related dialogs.
    • management: Management-related dialogs.
  • launcher: GUI launcher scripts.
  • modules: Different GUI modules.
  • utils: Utility scripts for GUI components.

Data Engine

The data_engine package handles the primary functionality for data processing and management within a project.

  • core: Core data processing modules.
  • data_streams: Handles live data input streams.
  • engine: Data engine implementations.
  • integration: Integration with external data sources, such as Amazon S3.
  • utils: Utility scripts for data handling.
  • variable_storage: Manages variable storage.
    • storage: Different storage backends for variables.

Analytics

The analytics package handles the application and visualization of data transformations within a project.

  • core: Core analytical processing and transformations.
  • custom_user: Custom user-defined transformations and configurations.
  • evaluation: Metrics and evaluation scripts for model performance.
  • models: Machine learning and statistical models.
  • prediction: Modules for making predictions using trained models.
  • preloaded: Preloaded transformations and configurations.
    • transformations: Preloaded transformation modules.
  • preprocessing: Data preprocessing modules.
  • training: Modules for training machine learning models.
  • utils: Utility functions and common metrics.
  • visualization: Display and visualization of analytical transformations.

(back to top)

Usage

To be implemented at a later date.

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create.

Open-source contributions will be available in the near future. Star & watch the project to stay tuned for updates!

(back to top)

License

Distributed under the BSD-3-Clause License. See LICENSE for more information.

(back to top)

Contact

Lane (B.S.)

Neurobiological Research Technician
Gire Lab, University of Washington
email: justlane@uw.edu


Dr. David H. Gire (Ph.D.)

Associate Professor, Principal Investigator
Gire Lab, University of Washington
email: dhgire@uw.edu





Project Link: https://github.com/lane-neuro/research-analytics-suite

(back to top)