Interactive 3D Avatar Setup in UE5

About the project

Project Title: Real-Time Interactive MetaHuman / Avatar Setup for UE5 (Cloud AI Voice Integrated) 1. Project Overview We are building a real-time interactive 3D digital human (MetaHuman) inside Unreal Engine 5 that acts as a live, streaming avatar. The system connects to a cloud-based AI backend (STT > Gemini LLM > ElevenLabs TTS over WebSockets). Your job is to set up the avatar, lighting, shaders, and real-time audio-driven animation pipeline in Unreal Engine 5 to achieve maximum visual photorealism at 60 FPS. 2. Deliverables & Technical Scope A. MetaHuman & Asset OptimizationCharacter Setup: Import, configure, and refine a MetaHuman (or custom high-fidelity 3D mesh mapped to the MetaHuman skeletal structure). Level of Detail (LOD): Lock the main avatar to LOD 0 (Cinematic).Groom / Hair: Implement high-quality strand-based groom hair optimized to hold 60 FPS.Shaders & Skin: Fine-tune skin materials (Subsurface Scattering profiles, micro-roughness, specular maps, eye reflections, and teeth/tongue moisture shaders). B. Real-Time Lighting & Cinematic SceneLighting Environment: Build a 3-point studio or natural lighting rig using Lumen with Hardware Ray Tracing.Camera Setup: Cinematic virtual camera (35mm / 50mm lens) with depth-of-field (DOF), motion blur tuned for live streaming, and auto-exposure locks.Performance Target: Scene must run at 4K / 60 FPS on a single NVIDIA RTX 4070 Ti Super / RTX 4080 (DLSS Quality enabled). C. Real-Time Animation & Audio Sync (Critical)Audio-Driven Lip Sync: Set up the NVIDIA Audio2Face-3D UE5 Plugin (or Live Link / ARKit blendshape pipeline) to drive facial expressions directly from incoming audio streams.Idle & Micro-Expressions: Implement a Blueprint state machine for idle animations (breathing, eye blinks, subtle head sways, micro-gestures) so the avatar stays "alive" while waiting for or listening to the user.Audio Routing: Configure incoming streaming audio (PCM / WAV chunks from ElevenLabs) to drive facial animation weights and spatial audio playback simultaneously. 3. Required Skills & Experience High proficiency in Unreal Engine 5 (5.4+), MetaHuman Creator, and Live Link protocols.Experience with NVIDIA Audio2Face-3D / ACE or ARKit blendshape mappings.Deep knowledge of real-time lighting (Lumen/Ray Tracing) and high-fidelity shader creation.Understanding of Blueprint / C++ networking for accepting external audio streams and animation trigger events.

Skills required