In collaboration with Blockade Labs, Intel Labs has launched the Latent Diffusion Model for 3D (LDM3D), a story model that uses AI to create 3D visuals.
LDM3D is the industry’s first type of depth mapping using a combination of 3D imaging and 360-degree imaging.
LDM3D is designed to revolutionize the production, use and experience of computing, revolutionizing a variety of industries, from entertainment and gaming to architecture and design.
Vasudev Lal, AI/ML Research Scientist at Intel Labs, says: “Generative AI technology aims to improve human performance while saving time.”
“However, most of today’s AI models are limited to creating 2D images, and only a few can create 3D images from stimulus words.”
“Unlike existing standard models, LDM3D allows users to create an image and depth map from a given image using nearly identical parameters.”
“It provides accurate depth for every pixel in the image compared to later methods of depth estimation and saves architects the time needed to create images,” says Lal.
Intel’s commitment to democratizing AI aims to provide access to the benefits of AI through an open environment.
Unlike existing models, which usually generate 2D RGB images from data, LDM3D allows users to create image and depth maps from the provided data.
By using the same number of parameters as a fixed diffusion, LDM3D provides a more accurate depth for each pixel in the image compared to standard methods after correcting depth calculations.
Images and depth maps created by LDM3D enable users to transform descriptions of a tranquil tropical beach, a modern skyscraper or a sci-fi universe into a detailed 360-degree panorama.
The ability to capture depth information can enhance virtual reality and instant immersion, enabling applications in industries ranging from entertainment and gaming to interior design and real estate listings, as well as museums and virtual reality (VR) experiences.
LDM3D was trained on a dataset made from 10,000 subsets of the LAION-400M database, which contains more than 400 million image pairs. The team described the training class using a Dense Prediction Transformer (DPT) model of deep learning (previously developed at Intel Labs).
The LDM3D model is trained on an Intel AI computer powered by Intel Xeon processors and Intel Habana Gaudi AI accelerators. The result and pipeline combine RGB images with depth mapping to create 360-degree views for immersive experiences.
To demonstrate the capabilities of LDM3D, Intel and Blockade researchers created DepthFusion, a program that uses 2D RGB images and depth maps to create immersive 360-degree visuals.
DepthFusion uses TouchDesigner, a real-time visual programming language, to transform audio into interactive, immersive digital experiences.
The introduction of LDM3D and DepthFusion provides a step forward in the field of artificial intelligence (AI) and computer vision.
Intel will continue to focus on the use of artificial intelligence to enhance human creativity and create a powerful ecosystem for open AI research and development that democratizes this technology.
LDM3D is being unlocked through HuggingFace, allowing AI researchers and experts to improve the system and optimize it for use.
#Intel #Labs #introducing #version #Diffusion
