aron/awebsite
aron/awebsite
1
0
Fork 0
Code Issues Pull requests Wiki Activity Actions

more translation

Browse source
This commit is contained in:
Aron Petau 2025-10-06 15:04:55 +02:00
parent 8ee055bfa8
commit 2ce9ca50b5
459 changed files with 21001 additions and 9395 deletions
Download patch file Download diff file Expand all files Collapse all files

28
content/project/2022-12-03-stable-dreamfusion/index.md
View file

@ -28,30 +28,30 @@ show_shares = true
## Sources
I forked a really popular implementation that reverse engineered the Google Dreamfusion algorithm. This algorithm is closed-source and not publicly available.
The implementation I forked is [here](https://github.com/arontaupe/stable-dreamfusion)
This one is running on stable-diffusion as a bas process, which means we are are expected to have worse results than google.
The original implementation is [here](https://dreamfusion3d.github.io)
I forked a popular implementation that reverse-engineered the Google Dreamfusion algorithm. This algorithm is closed-source and not publicly available.
You can find my forked implementation [on my GitHub repository](https://github.com/arontaupe/stable-dreamfusion).
This version runs on Stable Diffusion as its base process, which means we can expect results that might not match Google's quality.
The [original DreamFusion paper and implementation](https://dreamfusion3d.github.io) provides more details about the technique.
{{ youtube(id="shW_Jh728yg") }}
## Gradio
The reason i forked the code is so that i could implement my own gradio interface for the algorithm. Gradio is a great tool for quickly building interfaces for machine learning models. No code involves, any user can state their wish, and the mechanism will spit out a ready-to-be-rigged model (obj file)
I forked the code to implement my own Gradio interface for the algorithm. Gradio is a great tool for quickly building interfaces for machine learning models. No coding is required for the end user - they can simply state their wish, and the system will generate a ready-to-be-rigged 3D model (OBJ file).
## Mixamo
I used Mixamo to rig the model. It is a great tool for rigging and animating models. But before everything, it is simple. as long as you have a model with a decent humanoid shape in something of a t-pose, you can rig it in seconds. Thats exactly what i did here.
I used Mixamo to rig the model. It's a powerful tool for rigging and animating models, but its main strength is simplicity. As long as you have a model with a reasonable humanoid shape in a T-pose, you can rig it in seconds. That's exactly what I did here.
## Unity
I used Unity to render the model to the magic leap 1.
Through this, i could create an interactive and immersive environment with the generated models.
I used Unity to render the model for the Magic Leap 1 headset.
This allowed me to create an interactive and immersive environment with the generated models.
The dream was, to build a AI- Chamber of wishes.
You pick up the glasses, state your desires and then the algorithm will present to you an almost-real object in AR.
The vision was to build an AI Chamber of Wishes:
You put on the AR glasses, state your desires, and the algorithm presents you with an almost-real object in augmented reality.
Due to not having access to the proprietary sources from google and the beefy, but still not quite machine-learning ready computers we have at the studio, the results are not quite as good as i hoped.
But still, the results are quite interesting and i am happy with the outcome.
A single generated object in the Box takes roughly 20 minutes to generate.
Even then, the algorithm is quite particular and oftentimes will not generate anything coherent at all.
Due to not having access to Google's proprietary source code and the limitations of our studio computers (which, while powerful, aren't quite optimized for machine learning), the results weren't as refined as I had hoped.
Nevertheless, the results are fascinating, and I'm satisfied with the outcome.
A single object generation in the environment takes approximately 20 minutes.
The algorithm can be quite temperamental - it often struggles to generate coherent objects, but when it succeeds, the results are quite impressive.