You can't HANDLE ✋this model
So essentially,
URHand is a better relightable hand model for digital worlds
Paper: URHand: Universal Relightable Hands(15 pages)
Researchers from the Meta and Nanyang Technological University are interested in better photorealistic, personalized, and relightable hand ✋representation in a digital medium such as virtual reality.
Hmm..What’s the background?
The hand-capture process is computationally expensive and requires a production studio
Existing methods cannot often generalize to novel poses, illuminations, and identities, limiting their practical use
Relighting techniques can achieve impressive photorealism, but they are often limited in their ability to generalize to natural illuminations
Source: https://lexica.art/prompt/935aacf6-f179-4201-abc9-d338d34776bb
Ok, So what is proposed in the research paper?
Overall, current techniques face challenges in achieving a balance between generalization, fidelity, and computational efficiency for hand relighting. The proposed method in this paper addresses these issues by combining a spatially varying linear lighting model with a hybrid neural-physical learning framework. Here are some key elements of URHand:
It enables generalization to arbitrary illuminations and single-stage cross-identity training.
It combines the advantages of physically based rendering and data-driven appearance modeling.
It enables the rapid adaptation of the universal before a personalized use case from a phone scan. Their approach utilizes a single iPhone 12 to capture hand geometry and appearance for seamless integration into the URHand model.
And what’s next?
The researchers discuss several unresolved challenges. Currently, the quick personalization of URHand requires the complete mean texture of a target hand. Thus, it does not work with a single image. Future work would be inpainting the texture from a single image to enable single-view relightable hand reconstruction. As the URHand hand model is only driven by hand poses, it cannot capture appearance variations due to blood pressure or temperature changes. Researchers would like to investigate the use of URHand for other applications such as augmented reality, virtual try-on, and character animation.
So essentially,
URHand is a better relightable hand model for digital worlds