About Torchdyn

Project goals

The main objective of Torchdyn is to provide a centralized hub for layers, numerical routines and utilities required for high-performance implementation of numerical deep learning models

By providing a centralized, easy-to-access collection of model templates, tutorial and application notebooks, we hope to speed-up research in numerical deep learning and provide a hub of reusable computational primitives and numerical routines.

Torchdyn leverages modern PyTorch best practices and handles training with pytorch-lightning. We build Graph Neural ODEs utilizing the Graph Neural Networks (GNNs) API of dgl. For a complete list of references, check pyproject.toml. We offer a suite of ODE solvers, sensitivity methods, root finding algorithms and other common utilities for numerical deep learning.

This video provides an introduction to key concepts and potential applications.

Research

Interest in the blend of differential equations, deep learning and dynamical systems has been reignited by recent works

We explore how differentiable programming can unlock the effectiveness of deep learning to accelerate progress across scientific domains, including control, fluid dynamics and in general prediction of complex dynamical systems. Conversely, we focus on models powered by numerical methods and signal processing to advance the state of AI in classical domains such as vision of natural language.

Michael Poli, Stefano Massaroli, Luca Scimeca, Sanghyuk Chun, Seong Joon Oh, Atsushi Yamashita, Hajime Asama, Jinkyoo Park, Animesh Garg
Neural Hybrid Automata: Learning Dynamics With Multiple Modes and Stochastic Transitions

8 Jun 2021

arXiv:2106.04165 [cs.LG] (or arXiv:2106.04165v1 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2106.04165

Effective control and prediction of dynamical systems require appropriate handling of continuous-time and discrete, event-triggered processes. Stochastic hybrid systems (SHSs), common across engineering domains, provide a formalism for dynamical systems subject to discrete, possibly stochastic, state jumps and multi-modal continuous-time flows. Despite the versatility and importance of SHSs across applications, a general procedure for the explicit learning of both discrete events and multi-mode continuous dynamics remains an open problem. This work introduces Neural Hybrid Automata (NHAs), a recipe for learning SHS dynamics without a priori knowledge on the number, mode parameters, and inter-modal transition dynamics. NHAs provide a systematic inference method based on normalizing flows, neural differential equations, and self-supervision. We showcase NHAs on several tasks, including mode recovery and flow learning in systems with stochastic transitions, and end-to-end learning of hierarchical robot controllers.

Access our archive

More research

Who we are

Core team

  • Profile image of Stefano Massaroli

    Stefano Massaroli

    Postdoctoral Research Fellow at Mila - Quebec under Prof. Yoshua Bengio. Co-founder of @DiffeqML and Syntensor contributor. Interested in dynamical systems, deep learning, optimization and control.

  • Profile image of Michael Poli

    Michael Poli

    C. S. PhD Student at Stanford. Co-founder of @DiffeqML and Syntensor contributor. Working at the intersection of deep learning, generative models and numerical optimization.

  • Profile image of Clayton Rabideau

    Clayton Rabideau

    Cofounder and CEO/CTO at Syntensor. Applying geometric deep learning models and neural differential equations to biology, modeled as a dynamical system at the ‘edge of chaos’.

  • Profile image of Archis Joglekar

    Archis Joglekar

    Founding Machine Learning Engineer at Syntensor. Working on fundamental methods development in numerical solving suites and multi-scale dynamical systems.

Work with us

Syntensor

We’re hiring! Syntensor is combining methods in geometric deep learning with stochastic neural graph differential equations to model dynamic biological systems at scale. We believe their work is the most advanced commercial application of Torchdyn at scale. If this sounds intriguing, please send your resumé to talent@syntensor.com - speak soon!

Find out more about our work

Syntensor.com