About Torchdyn
Project goals
The main objective of Torchdyn is to provide a centralized hub for layers, numerical routines and utilities required for high-performance implementation of numerical deep learning models
By providing a centralized, easy-to-access collection of model templates, tutorial and application notebooks, we hope to speed-up research in numerical deep learning and provide a hub of reusable computational primitives and numerical routines.
Torchdyn leverages modern PyTorch best practices and handles training with pytorch-lightning. We build Graph Neural ODEs utilizing the Graph Neural Networks (GNNs) API of dgl. For a complete list of references, check pyproject.toml. We offer a suite of ODE solvers, sensitivity methods, root finding algorithms and other common utilities for numerical deep learning.
This video provides an introduction to key concepts and potential applications.
Research
Interest in the blend of differential equations, deep learning and dynamical systems has been reignited by recent works
We explore how differentiable programming can unlock the effectiveness of deep learning to accelerate progress across scientific domains, including control, fluid dynamics and in general prediction of complex dynamical systems. Conversely, we focus on models powered by numerical methods and signal processing to advance the state of AI in classical domains such as vision of natural language.
Michael Poli, Stefano Massaroli, Luca Scimeca, Sanghyuk Chun, Seong Joon Oh, Atsushi Yamashita, Hajime Asama, Jinkyoo Park, Animesh Garg
Neural Hybrid Automata: Learning Dynamics With Multiple Modes and Stochastic TransitionsEffective control and prediction of dynamical systems require appropriate handling of continuous-time and discrete, event-triggered processes. Stochastic hybrid systems (SHSs), common across engineering domains, provide a formalism for dynamical systems subject to discrete, possibly stochastic, state jumps and multi-modal continuous-time flows. Despite the versatility and importance of SHSs across applications, a general procedure for the explicit learning of both discrete events and multi-mode continuous dynamics remains an open problem. This work introduces Neural Hybrid Automata (NHAs), a recipe for learning SHS dynamics without a priori knowledge on the number, mode parameters, and inter-modal transition dynamics. NHAs provide a systematic inference method based on normalizing flows, neural differential equations, and self-supervision. We showcase NHAs on several tasks, including mode recovery and flow learning in systems with stochastic transitions, and end-to-end learning of hierarchical robot controllers.
Access our archive
More researchWho we are
Core team
Stefano Massaroli
Postdoctoral Research Fellow at Mila - Quebec under Prof. Yoshua Bengio. Co-founder of @DiffeqML and Syntensor contributor. Interested in dynamical systems, deep learning, optimization and control.
Michael Poli
C. S. PhD Student at Stanford. Co-founder of @DiffeqML and Syntensor contributor. Working at the intersection of deep learning, generative models and numerical optimization.
Clayton Rabideau
Cofounder and CEO/CTO at Syntensor. Applying geometric deep learning models and neural differential equations to biology, modeled as a dynamical system at the ‘edge of chaos’.
Archis Joglekar
Founding Machine Learning Engineer at Syntensor. Working on fundamental methods development in numerical solving suites and multi-scale dynamical systems.
Work with us
We’re hiring! Syntensor is combining methods in geometric deep learning with stochastic neural graph differential equations to model dynamic biological systems at scale. We believe their work is the most advanced commercial application of Torchdyn at scale. If this sounds intriguing, please send your resumé to talent@syntensor.com - speak soon!