ImJoy - Deploying Deep Learning Made Easy!
ImJoy is a plugin powered hybrid computing platform for deploying deep learning applications such as advanced image analysis tools.
ImJoy runs on mobile and desktop environment cross different operating systems, plugins can run in the browser, localhost, remote and cloud servers.
With ImJoy, delivering Deep Learning tools to the end users is simple and easy thanks to its flexible plugin system and sharable plugin URL. Developer can easily add rich and interactive web interfaces to existing Python code.
Checkout the documentation for how to get started and more details for how to develop ImJoy plugins: ImJoy Docs
Key Features of ImJoy
Minimal and flexible plugin powered web application
Serverless solution with offline support
Support mobile devices
Rich and interactive user interface powered by web technologies
- use any existing web design libraries
- Rendering multi-dimensional data in 3D with webGL, Three.js etc.
Easy-to-use workflow composition
Isolated workspaces for grouping plugins
Self-contained plugin prototyping and development
- Built-in code editor, no extra IDE is needed for development
Powerful computational backend powered by the Python ecosystem
- Concurrent plugin execution through asynchronous programming
- Run Python plugins in the browser with Webassembly
- Browser plugins are isolated with secured sandboxes
- Support Conda virtual environments and pip packages for Python
- Easy plugin deployment and sharing through GitHub or Gist
- Deploying your own plugin repository to Github
Native support for n-dimensional arrays and tensors
- Support ndarrays from Numpy for data exchange
- Support Tensorflow.js and native Tensorflow for deep learning
The easiest way to try ImJoy is to start with a plugin running directly in the browser.
For example, you can install our
Image Recongnition plugin with this link. Clicking the link will open the ImJoy Web App and show a dialog to confirm the installation of the plugin. Once installed, you can launch the plugin by clicking the button
Image Recognition in the plugin menu on the left. The demo plugin will run a trained deep neural network model (MobileNet) to perform image classification (e.g. identifying an elephant in an image).
This plugin uses your browser as its computational backend, so all computation is done locally, no data will be sent to a remote server.
Besides running plugins in the browser, ImJoy provides the flexibility to keep the GUI locally in your browser, and perform compuational tasks with Python. These computations can be performed on your computer utilizing the full power of the local GPU/CPU. The computional backend can also be launched on remote servers including cloud servers on Amazon, Google Compute, or an institutional computing cluster.
To learn more details about ImJoy, please go to ImJoy Docs.
By design, ImJoy itself provides very minimal task specific functionality. It provides a flexible plugin interface to support plugins to extend the user interface and perform computational tasks. Plugins can be designed to perform simple tasks such as reading tif file, or more complex tasks such as training a deep learning model for image segmentation.
Imjoy consists of three parts, each part can be extended with plugins:
Web User Interface. ImJoy is a web application. This means that using ImJoy is as easy as opening a web page from ImJoy.io. With ubiquitousness of the web, ImJoy provides unified user experience with browsers running on different operating systems, including mobile devices. The user can install plugins, and use them from the web interface. Besides a easy-to-use minimal interface, plugins can create their own window to create rich and interactive web interface by making use of the full power of the entire HTML/CSS/JS frameworks. For example, with D3.js or Three.js, one can easily provide powerful interactive charts or 3D visualization plugins. For developers, the web interface is also used for writing and testing code. A stand-alone desktop application (desktopApp) is also provided.
Native Computational Backend. This is supported with the installation of an additional Python module called "ImJoy Plugin Engine". This allows to access the entire Python ecosystem which covers most of the scientific computing applications. With Conda, ImJoy plugin engine handles the requirements of plugins automatically and provide isolate processes and virtual environments for different Python plugins. It provides maximum flexibility and has full access to the file system, GPU and other local or remote resources. The plugin engine can be launched either on the local machine, or remotely on a cloud server or a cluster to perform computationally intensive tasks, e.g. with institutional computing cluster, Amazon Cloud, or Google Compute.
Currently, ImJoy consists of the following repositories:
ImJoy Web App (core) (this repository)
Documentation, questions and feedback
- We provide a detailed ImJoy Docs. This includes a section with frequently asked questions.
- For further help and questions for how to use ImJoy and process data with ImJoy, you can post on image.sc.
- To report bugs or suggest enhancements for ImJoy itself, please file a GitHub issue.
This is a non-exhaustive list of the open-source tools and libraries we used in ImJoy:
- Joy.js (This is where the name
- Jailed (This helps ImJoy to isolate plugins)
- Vue.js (The main ImJoy App is written in Vue.js)
- vue-grid-layout (For window management)
- python-socketio (This how the plugin engine talk with the ImJoy app)
- pyodide (Enables web python mode with WebAssembly)
- conda (The plugin engine uses Conda for managing virtual env. and packages)
- docsify (The ImJoy documentation is created with docsify)
We thank all the authors for offering these excellent work to the open-source community.
Please star the ImJoy GitHub repository to support ImJoy.
We would like ImJoy to be a community driven framework, everyone is welcome to contribute your idea, feedback, plugins and code to the project.
We don't have a guidline for that yet, for now please feel free to use the issues and fork the project.