The world of streaming is about to be revolutionized by NVIDIA Eye Contact AI. The Nvidia Eye Contact AI feature is a new addition to the Nvidia Broadcast 1.4 software that leverages artificial intelligence (AI) to mimic natural eye contact during recording. If you become distracted during filming to look away to read notes or follow a script, the AI-powered feature will immediately make real-time adjustments to your video, making it appear as if you are maintaining direct eye contact with the camera.
Article structure:
Using NVIDIA Eye Contact AI your eyes look at the camera all the time
Mitigating the eye gaze problem in a typical desktop or laptop home setup is challenging, and quick fixes are mostly gimmicky and insufficient. However, some companies offer external webcams with a compact design and no integrated microphone, allowing you to place them in the center of your screen, aligned with the viewing window or any text material on it.
These webcams have a thin wire that clips to the top of the display, enabling you to look directly at the camera and see most of the presentation or text material you are sharing, although not all of it.
Nvidia has recently launched a beta version of Eye Contact, a video feature that leverages AI to automatically maintain eye contact during on-camera conversations by aligning gaze. Eye Contact is included in the 1.4 version of the company’s Broadcast app.
To achieve this effect, Eye Contact replaces your eyes in the video stream with software-generated eyes that mimic natural eye color and even blink when you do. These simulated eyes are designed to stare directly into the camera at all times, even if you are not looking directly at it.
Should you use NVIDIA Eye Contact AI feature?
The Eye Contact feature in Nvidia’s Broadcast app is currently in beta form and may have some glitches that need to be resolved before the production version is released.
There has been some adverse reaction to the use of AI to correct eye contact, with some labeling it as creepy. However, similar enhancements like filters on social media apps exist and have not received the same negative criticism.
In the post-pandemic world, videoconferencing has become a crucial tool, and utilizing technology to enhance its productivity and impact should be embraced.
Therefore, while the Eye Contact feature is in beta form and may need some improvements, the NVIDIA Eye Contact feature is definitely worth using. The feature allows your eyes look at the the camera all the time, even when you’re not directly looking at the camera, which can significantly improve communication with the person on the other end.
The feature to be incredibly useful and convenient, particularly for long virtual meetings or video calls where constantly looking at the camera can be tiring. This feature is especially beneficial for those who aren’t particularly fond of virtual meetings.
Whether you’re using Microsoft Teams or Zoom, this feature is available for use on your laptop or PC, making it a great tool for improving your virtual communication skills.
Why your eyes look at the camera all the time using NVIDIA Eye Contact AI?
NVIDIA Maxine Eye Contact employs the eye patch, a region of interest around the eyes, to adjust gaze. First, the eye patch is extracted from a video frame using NVIDIA Maxine Face tracking pipeline, which identifies 2D facial landmarks and 6DOF head pose from the video frame.
Next, the head pose is used to normalize the face in the video frame. A 256 ✕ 64 px eye patch is then cropped from the normalized frame and inputted into the eye contact network, which has a disentangled encoder-decoder architecture. The encoder estimates the gaze angle from the eye patch and generates a set of embeddings or features.
Using these embeddings, the decoder redirects the gaze in the eye patch to make it appear as if the user is looking directly into the camera. Finally, the eye patch is blended back into the original video frame using an inverse transformation.
The pipeline outputs the head pose, gaze angles, and image with redirected eye gaze. The pipeline can also be used for gaze estimation only, with redirection turned off.
The technology ensures that the redirection effect reduces as the original eye gaze drifts further away from the center, providing a natural-looking experience. Redirection is turned off if the head is rotated beyond a predetermined threshold, outside of which natural-looking redirection is not feasible.
Good to know: Synthesia: Automatic video creation with AI
How use the Nvidia Eye Contact AI feature
To use the Nvidia Eye Contact AI feature, follow these simple steps:
- Download Nvidia Broadcast if you haven’t already done so. You can find the download link on their website
- Navigate to the Camera section within the app
- Click on the Effect option
- Choose the Eye Contact (Beta) feature from the list of available effects.
With these steps completed, you can now use the Eye Contact AI feature in Nvidia Broadcast to simulate natural eye contact with the camera during recordings.
To use the NVIDIA Eye Contact AI feature, your system must meet the following requirements:
- At least 8GB of RAM
- GeForce RTX 2060 graphics card (or a Quadro-compatible GPU)
- Intel Core i5-8600 or AMD Ryzen 5 2600 processor at a minimum.
If your system meets these requirements, you’re ready to explore and utilize the NVIDIA Eye Contact AI feature.
What are the compatible applications for Nvidia Broadcast?
According to Nvidia, Broadcast is a versatile app that can work seamlessly with a range of software, including popular streaming software such as OBS and Streamlabs, as well as video conferencing apps like Skype, Zoom, and Teams. It can also be integrated with Discord and other applications.
About Nvidia Broadcast
Nvidia Broadcast is an app that could fulfill the streaming community’s wishes. It can improve the quality of your content without the need to upgrade your hardware. With your current microphone and webcam, Nvidia Broadcast can enhance your voice capture and give your live stream a more professional touch.
The app offers various useful features including:
- Auto framing: The app uses AI to track your movements, following you around the room if you move within the field of view. This feature ensures you are always at the center of the action, giving you the feeling of having your own personal cameraperson.
- Virtual backgrounds and blurring: Similar to Zoom and Skype, Nvidia Broadcast lets you adjust the background of your camera view. You can blur your background or add custom virtual backgrounds. You can even create green screen effects without a green screen, which means you can overlay your camera feed over game footage without the need for additional equipment.
- Noise removal: The app includes an improved version of RTX Voice, which uses AI to remove background and environmental noise from your microphone. This feature makes your voice clearer and your stream more professional, whether it’s dogs barking in the distance or the hum of your air conditioning.
