Integrate AI and underwater photography to reveal the hidden ocean world | MIT News

In the northeastern United States, the Gulf of Maine is one of the most biologically diverse marine ecosystems on the planet, home to whales, sharks, jellyfish, herring, plankton and hundreds of other species. However, despite the support of this ecosystem’s rich biodiversity, it is undergoing rapid environmental change. The Gulf of Maine is warming faster than 99% of the world’s oceans, and the results are still unfolding.

A new research initiative developed at MIT Sea Grant, called Lobstger, shortly to learn marine biological ecological systems through generative representations, linking artificial intelligence and underwater photography to document marine life vulnerable to these changes and share it with the public new visually. Co-led by MIT Sea Grant Keith Ellenbogen and MIT Mechanical Engineering PhD student Andreas Mentzelopoulos and visiting artists, the project explores how the generated AI can expand scientific storytelling by constructing field-based photo data.

Just as 19th century cameras changed their ability to document and reveal the natural world – capturing life with unprecedented details and bringing views into the distance or hidden environment – Generation AI marks a new frontier of visual storytelling. Like early photography, AI opens up creative and conceptual spaces, challenging how to define reliability and how to communicate scientific and artistic perspectives.

In the Lobstger project, the generative model is trained only with a curated library of each image (each image) made with artistic intent, technical accuracy, accurate species identification, and clear geographical context. By building high-quality datasets based on real-world observations, the project ensures that the resulting images maintain both visual integrity and ecological relevance. Additionally, Lobstger’s models are built using custom code developed by Mentzelopoulos to protect processes and output from potential biases from external data or models. Lobstger Generation AI is built on real photography to expand the researcher’s visual vocabulary and deepen the national connection with the natural world.

At its heart, Robsgar operates at the intersection of art, science and technology. This project is drawn from the visual language of photography, the observational rigor of marine science, and the computational power of generated AI. By integrating these areas, the team is not only developing new ways to visualize marine life, but also rethinking how environmental narratives are told. This integrated approach makes Lobstger both a research tool and a creative experiment. This reflects MIT’s longstanding tradition of interdisciplinary innovation.

Underwater photography in New England’s coastal waters is notoriously difficult. Limited vision, swirling sediments, bubbles, and unpredictable movements of marine life all pose constant challenges. Over the past few years, Elenbogen has navigated these challenges and has built a comprehensive record of biodiversity in the region through its project Space to Sea: Visualization of the Marine Wilderness in New England. This large dataset of underwater images provides the foundation for training Lobstger’s generated AI models. Images span a variety of angles, lighting conditions and animal behavior, resulting in artistically impressive and biologically accurate visual archives.

Lobstger’s custom diffusion model is trained to replicate not only the Ellenbogen document of biodiversity, but also the artistic style he uses to capture it. By learning from thousands of real underwater images, the model internalizes fine grain details, such as natural lighting gradients, species-specific colors, and even atmospheric textures created by suspended particles and refracted sunlight. The results are images that are visually accurate, but also immersive and moving.

The model can unconditionally generate new, synthetic, scientifically accurate images (i.e., no user input/guidance required), and enhance conditional (i.e., image-to-image generation). By integrating AI into your photography workflow, Ellenbogen can use these tools to recover details in turbulent water, adjust lighting to highlight important subjects, and simulate scenes that are almost impossible to capture in the field. The team also believes this approach could benefit other underwater photographers and image editors facing similar challenges. This hybrid approach is designed to accelerate the curation process and allow storytellers to build more complete, consistent visual narratives beneath the surface.

In one important series, Ellenbogen took high-resolution images of lion mane jellyfish, blue sharks, American lobsters and marine sunfish (Mora Mora) Free diving in coastal waters. “Getting a high-quality dataset is not easy,” says Ellenbogen. “It requires multiple dives, missed opportunities, and unpredictable conditions. But these challenges are part of what makes underwater documentation difficult and rewarding.”

Mentzelopoulos developed original code to train a family of potential spreading models of Lobstger grounded to an Ellenbogen image. Developing such models requires a high level of technical expertise, and training models from scratch are complex processes that require hundreds of hours of calculation and thorough hyperparameter tuning.

This project reflects parallel processes: field documentation through photography and iterative training through model development. Elenbogen works in this field, capturing rare and fleeting encounters with marine animals. Mentzelopoulos works in the lab and transforms those moments into machine learning contexts that can be extended and reinterpreted by the visual language of the ocean.

“The goal is not to replace photos,” says Mentzelopoulos. “It’s about building it and complementing it. It creates invisible visible things and helps people see the complexity of the environment in a way that resonates emotionally and intellectually. Our model aims to capture not only biological realism, but emotional charging that can promote real-world engagement and behavior.”

Lobstger points to a hybrid future that integrates technical interpretations with direct observations. The team’s long-term goal is to develop a comprehensive model that can visualize the wide range of species found in the Gulf of Maine, and ultimately apply similar methods to marine ecosystems around the world.

Researchers suggest that photography and the generation AI form a continuum rather than conflict. While photography captures textures, lights, and animal behavior during real encounters, AI expands its vision beyond what is seen, beyond what is seen, towards what can be understood, guessed or imagined. Together, they provide a powerful framework for communicating science through imagemaking.

In areas where ecosystems are changing rapidly, the act of visualization becomes more than just a document. It becomes a tool for recognition, engagement and ultimately conservation. Lobstger is still in its early stages and the team looks forward to sharing more discoveries, images and insights as the project evolves.

Answer from lead image: The left image is generated using Lobstger’s unconditional model, and the right image is real.

For more information, contact Keith Ellenbogen and Andreas Mentzelopoulos.