Rotating an image of a molecular structure allows humans to know that the rotated image is still the same molecule, but machine learning models may think that it is a new data point. In computer science terminology, a molecule is “symmetric”, and the basic structure of that molecule remains the same when undergoing a specific transformation, such as rotation.
If the drug discovery model does not understand symmetry, it may make inaccurate predictions about molecular properties. However, despite some empirical success, it was unclear whether there was a computationally efficient way to train good models that are guaranteed to respect symmetry.
A new study by MIT researchers answers this question and presents the first method of machine learning with empirically efficient symmetry in both the complexity of computation and the data required.
These results can clarify basic questions and help researchers develop stronger machine learning models designed to handle symmetry. Such models can be useful in a variety of applications, from discovering new materials to identifying astronomical anomalies to elucidating complex climate patterns.
“These symmetry is important because it’s a kind of information that nature speaks about data and you need to consider it in your machine learning model. Now we’ve shown that machine learning can be done efficiently with symmetric data.”
He joined the paper with co-starring author Ashkan Soleymani, a graduate student at MIT. Stefanie Jegelka is an associate professor in Electrical Engineering and Computer Science (EECS) and is a member of the Data, Systems, Society, Society (IDSS) and the Institute for Computer Science and Artificial Intelligence (CSAIL). Senior author Patrick Gyrett, Professor of Electrical Engineering and Computer Science at Dugard C. Jackson, and Chief Investigator in the Information and Decision Systems (LID) Laboratory. The study was recently presented at an international conference on machine learning.
Symmetry studies
Symmetric data appears in many domains, especially natural sciences and physics. A model that recognizes symmetry can identify objects like cars, for example, regardless of where the object is located.
Unless machine learning models are designed to handle symmetry, they tend to be less accurate and break down when faced with new symmetric data in real-world situations. Conversely, models that exploit symmetry may be faster and require less data to train.
However, training a model to process symmetric data is not easy.
One common approach is called data augmentation, and researchers can transform each symmetric data point into multiple data points to help the model better generalize to new data. For example, it is possible to rotate molecular structures multiple times to create new training data, but this could be computationally prohibited if researchers wish to ensure that the model respects symmetry.
Another approach is to encode symmetry into the architecture of the model. A well-known example of this is Graph Neural Networks (GNN). This essentially processes symmetric data for the design method.
“Graph neural networks are fast, efficient and handle symmetry very well, but no one really knows what these models are learning or why they work. Understanding GNNS is the main motivation for our work, so we started with a theoretical assessment of what happens when the data is symmetric,” says Tahmasebi.
They investigated statistical compartmental trade-offs in machine learning using symmetric data. This trade-off means methods that require methods where less data can become more computationally expensive, so researchers need to find the right balance.
Based on this theoretical evaluation, researchers designed efficient algorithms for machine learning using symmetric data.
Mathematical combinations
To do this, they borrowed ideas from algebra to reduce and simplify the problem. We then reformulated the problem using ideas from geometry that effectively captures symmetry.
Finally, they combined algebra and geometry into optimization problems to bring about new algorithms, as they can be solved efficiently.
“Most of theories and applications focused on either algebra or geometry. Here we just combined them,” says Tahmasebi.
This algorithm provides fewer data samples for training than classical approaches, improving the accuracy and ability of the model to adapt to new applications.
By proving that scientists can develop efficient algorithms for symmetric machine learning and demonstrate how it is done, these results could lead to the development of new neural network architectures that are more accurate and resource-intensive than current models.
Scientists can also use this analysis as a starting point to explore the internal mechanisms of GNN. You can also look at how the algorithms developed by MIT researchers differ from their operations.
“When you know that better, you can design a more interpretable, more robust, and more efficient neural network architecture,” adds Soleymani.
This research is funded in part by the National Research Foundation of Singapore, the DSO National Institute of Singapore, the US Naval Research Office, the National Science Foundation, and Professor Alexander von Humboldt.
