Back in February, NVIDIA introduced the LLM-based chatbot Chat with RTX. In May, this chatbot received an update, introducing new models and features. The package size has been reduced from 35GB to 11GB, and the software has officially been renamed ChatRTX.
In the previous article and video about Chat with RTX, we mentioned that Chat with RTX does not come with built-in Chinese responses. To achieve Chinese responses, users need to manually install environments and large language models. However, this step has a relatively high barrier for users, as it involves many complex steps to enable Chinese question answering.
Before we dive into the introduction, let's briefly explain what ChatRTX is. ChatRTX utilizes RAG technology, powered by NVIDIA TensorRT-LLM and NVIDIARTX acceleration, to bring chatbot functionality to RTX Windows PCs and workstations. Therefore, the prerequisite for using ChatRTX is to have an RTX 30 or RTX 40 series graphics card with 8GB of VRAM or more.
The main feature of ChatRTX is its operation on local devices, unlike various AI chatbots that operate in the cloud. Local processing ensures better data security, and with the support of NVIDIA TensorRT-LLM, it also enables faster processing, avoiding situations where it takes ages to generate a response to input issues.
In this ChatRTX update, in addition to the original Gemma model, a new ChatGLM3 model that supports both Chinese and English has been introduced. This makes it more convenient to use as there is no need for complex setup like in the previous version. With this update, more users can easily get started, engage in seamless Chinese conversations, quickly search for the desired content in imported documents, and present it to the users.
In addition to the new large language model, ChatRTX now includes a feature for image prompt word retrieval. After importing a folder containing images, ChatRTX can extract keywords from the images using OpenAI CLIP. This means that if you input a keyword related to an image, such as "mountain climbing," ChatRTX will provide you with images from the folder related to mountain climbing. This greatly facilitates local image searches, allowing you to find images even if you have forgotten about them through keyword searches.
In addition, ChatRTX has also incorporated a voice recognition feature that can recognize speech within 30 seconds and input it into the dialogue box, including recognizing both Chinese and English. Whether it's freeing up hands during work or expanding functionality in the future, voice recognition is beneficial to users. However, a problem from the previous version still exists: ChatRTX is unable to maintain context, meaning it clears its memory after each question. Therefore, each query can only be a "one-time" question.
However, ChatRTX is expected to continue updating in the future, becoming a fast feedback localized chat and file retrieval bot. The prerequisite for all this is that you must have an NVIDIA RTX 30 series graphics card. As they say, "out with the old, in with the new," the recently released Inno3D RTX 4070 SUPER Metal Master OC, part of the new RTX 40 SUPER series graphics cards, is your best choice.
The Colorful RTX 4070 SUPER Metal Master OC features the NVIDIA Ada Lovelace architecture, delivering powerful performance with 12GB of VRAM to meet the hardware demands of LLM. Whether for gaming or AI applications, the Colorful RTX 4070 SUPER Metal Master OC excels. Fans of this product are welcome to visit the official Colorful store for selection and purchase.
