Phonemation

Phonemation is a cutting-edge Vtuber software that allows users to animate virtual avatars using only their voice.
Our software uses advanced AI algorithms to recognize phonemes and emotions from the user's voice and creates dynamic animations that correspond to the user's speech patterns.
With Phonemation, anyone can create high-quality virtual content for streaming, gaming, or other online activities.

A computer with a NVIDIA GPU with CUDA support is recommended!
But it can also run only with the CPU, it's recommended to turn off the AI features in that case to save performance.

Why Phonemation?

🚀 Real-time Animation: Phonemation captures your every word and emotion in real-time, infusing your virtual avatar with dynamic expressions, bringing a new dimension to your content.

🎨 Limitless Creativity: Customize your virtual persona with precision. From facial features to clothing styles, every detail can be tailored to reflect your unique personality and style.

🤖 Powered by AI: Our cutting-edge AI technology seamlessly recognizes phonemes and emotions from your voice, ensuring a lifelike and responsive virtual performance.

🌟 Versatile Integration: Whether you're a streamer, content creator, or looking to enhance your online conversations, Phonemation seamlessly integrates with your favorite platforms.

Experience Phonemation:

Elevate your content creation journey with Phonemation – no prior animation experience required. Let your voice be the brushstroke, and watch as your virtual avatar comes alive in real-time.

What's Included:

📦 Instant access to Phonemation software
📚 Comprehensive user guide
🌐 Exclusive access to our thriving community for insights and inspiration
🚀 Lifetime updates for free

Join the Revolution:

Embrace the future of content creation with Phonemation. Ignite your creativity, captivate your audience, and stand out in the Vtubing and content creation landscape.

Planned Changes / Additions:

• Improved handling of incorrect setups and missing files ✅
• Progress Information for downloads ✅
• Clearing download cache and deletion of AI models on demand
• Background and foreground layers which don't move with the character ✅
• Drag and Drop for Character folders and GLSL shaders ✅
• And many more improvements and additions ✅

Update 23 October 2023: A lot of updates have happened as you can see above.
Besides those there are also these changes/additions:

• Fixed installer, it is now much bigger but should correctly include CUDA versions
• Layer folders should now be in this format: layerzposition_layername, so for example you will have 2_bodies instead of only bodies like before.
• You can add foregrounds and backgrounds and activate and switch them with the toggle command.
• All layers can have emotion variants, for the eyes you can have open and closed variations with emotions in this format: emotion_state; So for example: angry_open or sad_closed.
• background_color can now be changed via the animation.yaml or the rest API.

Update 5 March 2024:

• Bug fix for Error on first Startup without a project set up and updated libraries.

Update 12 July 2024:

• Change to new AI Runtime, allows smaller file sizes.
• New Audio System allows using different Host APIs if available.
• New Packaging for smaller and faster program.
• Subscription based version, might replace old one. Allows cheaper testing.
• Many bug fixes.
• Note that this version seems to not see CUDA at the moment, I'm working on fixing this. Fixed!

Update 22 July 2024:

• Added some better Defaults and Fallbacks
• Added an example which can be used as a template
• Many more smaller bug fixes

Update 29 July 2024:

• File and Folder selection was not working, fixed.
• Improved the included Example Template
• Installer and Program now include better version information, currently at version 0.4.0

You very likey will need to install the CUDA (12.x) and cuDNN (9.x) Runtimes for the CUDA acceleration to work:
CUDA Runtime

cuDNN Runtime

(I will likely use the knowledge about these new AI features to add some improvements to Papagayo-NG too.)

For Documentation and Information about how to use Phonemation you can look in the Documentation here:
Phonemation Documentation