NVIDIA just launched TensorRT 7, introducing the capability for Real-Time Conversational AI!
Here is a primer on the NVIDIA TensorRT 7, and the new real-time conversational AI capability!
NVIDIA TensorRT 7 with Real-Time Conversational AI
NVIDIA TensorRT 7 is their seventh-generation inference software development kit. It introduces the capability for real-time conversational AI, opening the door for human-to-AI interactions.
TensorRT 7 features a new deep learning compiler designed to automatically optimise and accelerate the increasingly complex recurrent and transformer-based neural networks needed for AI speech applications.
This boosts the performance of conversational AI components by more than 10X, compared to running them on CPUs. This drives down the latency below the 300 millisecond (0.3 second) threshold considered necessary for real-time interactions.
TensorRT 7 Targets Recurrent Neural Networks
TensorRT 7 is designed to speed up AI models that are used to make predictions on time-series, sequence-data scenarios that use recurrent loop structures (RNNs).
RNNs are used not only for conversational AI speed networks, they also help with arrival time planning for cars and satellites, predictions of events in electronic medical records, financial asset forecasting and fraud detection.
The use of RNN has hitherto been limited to a few companies with the talent and manpower to hand-optimise the code to meet real-time performance requirements.
With TensorRT 7’s new deep learning compiler, developers now have the ability to automatically optimise these neural networks to deliver the best possible performance and lowest latencies.
The new compiler also optimises transformer-based models like BERT for natural language processing.
TensorRT 7 Availability
NVIDIA TensorRT 7 will be made available in the coming days for development and deployment for free to members of the NVIDIA Developer program.
The latest versions of plug-ins, parsers and samples are also available as open source from the TensorRT GitHub repository.
Recommended Reading
- NVIDIA DRIVE Deep Neural Networks : Access Granted!
- DiDi Adopts NVIDIA AI + GPUs For Self-Driving Cars!
- NVIDIA DRIVE AGX Orin for Autonomous Vehicles Revealed!
- Porsche Tri-Wing S-91x Pegasus Starfighter Revealed!
- NVIDIA Image Sharpening For DirectX, Vulkan + OpenGL
- Quake II RTX : What NVIDIA Changed In New Version 1.2!
- How To Enable NVIDIA NULL For G-SYNC Monitors Correctly!
- Learn How To Add ReShade Filters To GeForce Experience!
- GALAX GeForce GTX 1660 Super Graphics Card Review!
- NVIDIA GeForce GTX SUPER : Everything You Need To Know!
- NVIDIA GeForce GTX 1660 SUPER : The Full Details!
- NVIDIA Jetson Xavier NX : World’s Smallest AI Supercomputer
- NVIDIA Wins MLPerf Inference Benchmarks For DC + Edge!
- Facebook Introduces New Multi-Colour Brand Logo!
- Samsung – IBM AI IoT Cloud Platform For 5G Mobile Solutions!
- Dell PowerMax Updates : Storage Class Memory + NVMe-oF!
- The Carousell Rights Owners Programme (CROP) Explained!
- Samsung Exynos 990 – Everything You Need To Know!
- 10GB + 12GB Samsung LPDDR4X uMCP Memory Details!
- Samsung Exynos Modem 5123 – What You Need To Know!
- Dell Offers Two DFS Flexi Options In These Countries!
- How AMD CPUs Work In A Secured-core PC Device!
- The Microsoft Secured-core PC Initiative Explained!
- Key NVIDIA EGX Announcements @ MWC Los Angeles 2019!
- Yahoo Groups To Delete All User Content! Download ‘Em Now!
- Kingston DC500 SSDs Are Now Certified VMware Ready!
- Dell EMC PowerProtect DD for Multi-Cloud Data Protection
- NTT Mega Merger Continues In Malaysia, ASEAN + APAC!
- Red Hat Partners Are Driving APAC Hybrid Cloud Adoption!
- The Alibaba Hanguang 800 (含光 800) AI NPU Explained!
- 3rd Gen X-Dragon Architecture by Alibaba Cloud Explained!
Go Back To > Software | Business | Home
Support Tech ARP!
If you like our work, you can help support our work by visiting our sponsors, participating in the Tech ARP Forums, or even donating to our fund. Any help you can render is greatly appreciated!
