How to get started with 3D gen AI
Metaverse masterpiece: jump the queue and start creating digital assets for free using 3D gen AI tools, such as Luma’s GENIE.
|
Getting your Trinity Audio player ready...
|
With apps such as Stable Diffusion, DALL-E, and Midjourney having conquered the world of 2D text-to-image generation, it’s no surprise to see gen AI tools set their sights on three-dimensional objects. Again, users have plenty of 3D gen AI approaches to choose from, with developers such as Google, OpenAI, NVIDIA, and Tencent all active in the scene.
However, one text-to-3D image app that’s heating up in particular is Genie, which is available on Discord as a research preview. The free 3D gen AI app can be accessed by accepting a Discord invitation from Karan Ganesan – Founding Engineer at Luma AI, the company behind Genie.
Research preview: 3D gen AI accelerates the creation of digital assets and beginners can jump straight in.
Luma AI is one of a growing number of firms helping users to bring real-life objects into the metaverse through 3D capture software. One of its popular tools is dubbed Flythroughs, which was announced in August 2023 and takes viewers on an AI powered journey through a digitally photographed world as if it were filmed using a drone.
The ‘phone not drone’ technique will certainly shake up the world of virtual tours and gives anyone with an iPhone and the Luma Flythroughs app a shortcut to creating some amazing footage. And the 3D gen AI wonders don’t stop there, as anyone wanting to quickly create digital assets for their metaverse crib, or prototype an industrial sim environment, will want to check out Genie too.
Discord interface: you’ll soon get used to the UI. Features to note include the search bar (highlighted top right in yellow), which makes it easy to separate out your 3D digital assets from thousands of other user-prompted creations appearing in the five genie threads (see the blue box in the right panel). Simply enter your username into the search box to find your Luma Genie text-to-3D models and to check on the progress of any high-resolution renders.
How to create 3D assets using gen AI
Luma has a genie-prompt-guide on its Discord channel, which shows users how to get started with the text-to-3D tool. Arguably, step one is to start thinking about the assets that you want to place in a 3D digital world. And, once you have that list, to draft a list of short descriptions that will serve as text prompts for the Genie 3D gen AI tool.
For example, say you were building an executive office in the metaverse, you might want –
- A comfortable office chair
- A coffee machine
- A trophy cabinet full of awards
And instructing the Luma AI app is, starting with the first asset, as simple as typing –
/genie a comfortable office chair
There are five genie bots available. Select one of them and then key in your prompt, prefixed by
/genie
Putting the Genie text-to-image 3D gen AI app to the test, it took just a couple of seconds for Luma AI’s servers to respond with a selection of four rotating images to choose from. However, the response time will depend on the number of users currently accessing the app. And don’t forget that you can use the search bar to find your text-to-3D prompted creations.
3D gen AI – make your selection and refine
Buttons below the image let users view each of the four text-to-3D images in more detail. The order runs clockwise, starting top left, and once you have decided upon your favorite version, you can place the model into a queue to generate a final high-resolution render by clicking on the corresponding refine button.
Select and refine: each text-to-image prompt entered into any of five genie bot threads produces four 3D gen AI versions to choose from. And the final selection can be rendered in high-resolution by adding it to the ‘genie-refine’ workflow.
Donuts and teapots – the ‘hello world’ of 3D modeling
The ‘hello world’ equivalent in 3D modeling is to create a realistic-looking donut. Teapots are a popular starting point too. And it’s remarkable how quickly Genie’s 3D gen AI engine is able to complete both tasks.
Users can download models (which include textures) from the Luma Genie 3D gen AI tool as GLB files, with GLB being a more compact version of GL Transmission Format (glTF) files.
Fancy a bigger armrest on your virtual office chair? The GLB files downloadable from Luma AI’s Genie tool can be imported into Blender, where you can get to work sculpting the 3D gen AI model like a piece of digital clay.
Just the beginning: 3D gen AI models can be imported into Blender, where shapes can be sculpted like digital clay.
Spending just a couple of hours with Genie, and with no prior digital 3D modeling experience, you can come a long way fast thanks to the power of 3D gen AI. The first wave of text-to-image tools was mind-blowing in two dimensions, and the latest crop of text-to-3D apps, such as Genie, are next level.
What’s interesting too, is how the arrival of capable and beginner-friendly 3D gen AI apps coincides with the release of next-generation mixed reality VR headsets, such as the Quest 3. Speaking with VR app developers and visiting the labs of extended reality (XR) chip developers, it feels like 2024 could be a breakthrough year for enterprise VR.
The fact that you can learn real-world tasks in VR makes the metaverse a professionally interesting space for companies to get involved in. And now, creators of those worlds – which can be productive environments not just for virtual meetings, but to gain practical experience in lean manufacturing or operating heavy machinery, to give just a couple of examples – can very quickly add digital assets with a simple text prompt.
5 March 2024
