We are very proud to announce the world's first and only Nvidia GH200 Grace-Hopper Superchip-powered and Nvidia Grace-Grace Superchip-powered supercomputers in quiet, handy and beautiful desktop form factors. Our benchmarks show that they are currently by far the fastest AI and also the fastest ARM desktop PCs in the world. Soon, our line-up will also feature transportable systems with up to eight high-speed linked GH200 Grace-Hopper or Grace-Grace superchips.What are the main benefits of GH200 Grace-Hopper?Its performance in every regard is almost unreal (up to 284 times faster than x86).Much cheaper than alternative systems with the same amount of memory.It has enough memory to run the biggest LLMs currently available.Optimized for memory-intensive inference and HPC performance.Ideal for AI, especially inference and fine-tuning of LLMs.Ideal for HPC applications like, e.g. genome sequencing.Connect display and keyboard, and you are ready to go.You can use it as a server or a desktop/workstation.Easily customizable, upgradable and repairable.Privacy and independence from cloud providers.Cheaper and much faster than cloud providers. Flexibility and the possibility of offline use.Perfect for edge AI ML GPT LLM and HPC.Gigantic amounts of coherent memory.No special infrastructure is needed.The lowest possible latency.It is very power-efficient.It is easy to transport.It is very quiet.It is beautiful.Runs Linux. Configure
What is the difference to alternative systems with the same amount of memory?Compared to a 8x Nvidia H100 system, GH200 costs 5x less, consumes 10x less energy and has very roughly the same performance.Compared to a 8x Nvidia A100 system, GH200 costs 3x less, consumes 5x less energy and has at least the same performance.Compared to a 4x AMD Mi300X system, GH200 costs 2x less, consumes 4x less energy and has probably roughly the same performance.Compared to a 4x AMD Mi300A system (which has only 512 GB memory, more is not possible because the maximum number of scale-up infinity links is 4), GH200 costs significantly less, consumes 3x less energy and has probably at least the same performance.Compared to a 8x Nvidia RTX A6000 Ada system which has significantly less memory (only 384GB), GH200 costs significantly less, consumes 3x less energy and has a higher performance.Compared to a 8x AMD Radeon PRO W7900 system which has significantly less memory (only 384GB), GH200 costs the same, consumes 3x less energy and has a higher performance.
The alternative systems mentioned above also have one thing in common: they are not available in standard desktop form factors, like our GH200 systems are.
PS: Please note that because of lack of benchmark data, the assumptions above are very rough estimates based on publically available information and in house benchmarking. We partner with Phoronix to benchmark as much as possible and will hopefully soon have solid data in form of publically available benchmarks to see how the different solutions compare for different workloads. The comparisons are expected to vary greatly for different workloads. If you want to know how your workloads performs on GH200 you can apply for a remote bare metal test here: Try What is the difference to 19-inch server models?Form factor: 19-inch servers have a very distinct form factor. They are of low height and are very long, e.g. 438 x 87.5 x 900mm (17.24" x 3.44" x 35.43"). This makes them rather unsuitable to place them anywhere else than in a 19-inch rack. Our GH200 and Grace tower models have desktop form factors: 244 x 567 x 523 mm (20.6 x 9.6 x 22.3") or 255 x 565 x 530 mm (20.9 x 10 x 22.2") or 250 x 404 x 359 mm (9.8 x 15.9 x 14.1"). This makes it possible to place them almost anywhere.Noise: 19-inch servers are extremely loud. The average noise level is typically around 90 decibels, which is as loud as a subway train and exceeds the noise level that is considered safe for workers subject to long-term exposure. In contrast, our GH200 and Grace tower models are very quiet (factory setting is 25 decibels) and they can easily be adjusted to even lower or higher noise levels because each fan can be tuned individually and manually from 0 to 100% PWM duty cycle. Efficient cooling is ensured, because our GH200 tower models have a higher number of fans and the low-revving Noctua fans have a much bigger diameter compared to their 19-inch counterparts and move approximately the same amount of air or even a much higher amount depending on the specific configuration and PWM tuning.Transportability: 19-inch servers are not meant to be transported, consequently, they lack every feature in this regard. In addition, their form factor makes them rather unsuitable to be transported. Our GH200 tower models, in contrast, can be transported very easily. Our metal and mini cases even feature two handles, which makes moving them around very easy.Infrastructure: 19-inch servers typically need quite some infrastructure to be able to be deployed. At the very least, a 19-inch mounting rack is definitely required. Our GH200 models do not need any special infrastructure at all. They can be deployed quickly and easily almost everywhere.Latency: 19-inch servers are typically accessed via network. Because of this, there is always at least some latency. Our GH200 tower models can be used as desktops/workstations. In this use case, the latency is virtually non-existent. Looks: 19-inch server models are not particularly aesthetically pleasing. In contrast, our available case options are in our humble opinion quite beautiful.ConfigureTechnical details of our GH200 workstations (base configuration):Metal tower with two color choices: Titan grey and Champagne goldGlass tower with four color choices: white, black, green or turquoiseMini tower with two color choices: white and blackAvailable air or liquid-cooledNvidia GH200 Grace Hopper Superchip72-core Nvidia Grace CPUNvidia H100 Tensor Core GPU480GB of LPDDR5X memory with error-correction code (ECC)96GB of HBM3 or 144GB of HBM3e576GB or 624GB of fast-access memoryNVLink-C2C: 900 GB/s of coherent memoryProgrammable from 450W to 1000W TDP (CPU + GPU + memory)2x High-efficiency 2000W/2400W PSU2x PCIe gen4/5 M.2 22110/228 slots on board2x/4x/8x PCIe gen4/5 drive slots (NVMe)2x/3x FHFL PCIe Gen5 x161x/3x/4x USB 3.0 ports2x RJ45 10GbE ports1x RJ45 IPMI port1x Mini display portHalogen-free power cablesStainless steel boltsVery quiet, the factory setting is 25 decibels (fan speed and thus noise level can be individually and manually configured from 0 to 100% PWM duty cycle)2 years manufacturer's warranty244 x 567 x 523 mm (20.6 x 9.6 x 22.3") or 255 x 565 x 530 mm (20.9 x 10 x 22.2") or 250 x 404 x 359 mm (9.8 x 15.9 x 14.1")30 kg (66 lbs) or 20 kg (44 lbs)Optional components:NIC Nvidia Bluefield-3 400GbNIC Nvidia ConnectX-7 200GbNIC Intel 100GbWLAN + Bluetooth cardUp to 2x 8TB M.2 SSDUp to 8x 8TB E1.S SSDUp to 10x 30TB U.3 SSDAdditional USB portsMulti-display graphics cardAdditional Tensor-Core-GPUsSound cardMouseKeyboardConsumer or industrial fansIntrusion detectionOS preinstalledAnything possible on requestWhat are the main differences between the offered GH200 models:GH200: metal tower, air-cooled, with 1 of 2 M.2 and 1 of 4 E1.S hard disks, 3x USBGH200 Special Edition: metal tower, air-cooled, without M.2 (0 of 2) and 2.5" (0 of 2) hard disks, 1x USBGH200 Liquid: metal tower, liquid-cooled, comes with 1 of 2 M.2 and 1 of 8 E1.S hard disks, 4x USBGH200 Glass: glass tower, air-cooled, with 1 of 2 M.2 and 1 of 4 E1.S hard disks, 3x USBGH200 Glass Special Edition: glass tower, air-cooled, without M.2 (0 of 2) and 2.5" (0 of 2) hard disks, 1x USBGH200 Glass Liquid: glass tower, liquid-cooled, comes with 1 of 2 M.2 and 1 of 8 E1.S hard disks, 4x USBGH200 Mini: mini tower, air-cooled, not yet known number of hard disks and USBCompute performance:67 teraFLOPS FP64989 teraFLOPS TF321,979 teraFLOPS FP163,958 teraFLOPS FP83,958 TOPS INT8Benchmarks:Phoronix is currently benchmarking our GH200 576GB model prototype. Initial results are available here:
https://www.phoronix.com/review/aarch64-64k-kernel-perfExample use case: Inferencing Falcon-180B LLMDownload: https://huggingface.co/tiiuae/falcon-180BFalcon-180B is a 180 billion-parameters causal decoder-only model trained on 3,500B tokens of RefinedWeb enhanced with curated corpora.Why use Falcon-180B? It is one of the best open-access models currently available, and one of the best models overall. Falcon-180B outperforms LLaMA-2, StableLM, etc. It is made available under a permissive license allowing for commercial use.Falcon-180B needs at least 400GB of memory to swiftly run inference! Luckily, GH200 has a minimum of 576GB.White paper: Nvidia GH200 Grace-Hopper white paper
ConfigureThe Grace-Grace superchipWhite paper: Nvidia Grace-Grace white paper
Trademark information: Nvidia is a trademark of Nvidia corporation.
Here you can find various downloads concerning our GH200 and Grace systems: operating systems, firmware, drivers, software, manuals, white papers, spec sheets and so on. Everything you need to run your system and more.Spec sheetsGH200 576GB: Spec sheet GH200 576GB.pdfGH200 624GB: Spec sheet GH200 624GB.pdfGH200 Special Edition 576GB: Spec sheet GH200 Special Edition 576GB.pdfGH200 Special Edition 624GB: Spec sheet GH200 Special Edition 624GB.pdfGH200 Liquid 576GB: Spec sheet GH200 Liquid 576GB.pdfGH200 Liquid 624GB: Spec sheet GH200 Liquid 624GB.pdfGH200 Glass 576GB: Spec sheet GH200 Glass 576GB.pdfGH200 Glass 624GB: Spec sheet GH200 Glass 624GB.pdfGH200 Glass Special Edition 576GB: Spec sheet GH200 Glass Special Edition 576GB.pdfGH200 Glass Special Edition 624GB: Spec sheet GH200 Glass Special Edition 624GB.pdfGH200 Glass Liquid 576GB: Spec sheet GH200 Glass Liquid 576GB.pdfGH200 Glass Liquid 624GB: Spec sheet GH200 Glass Liquid 624GB.pdfGH200 Mini 576GB: Spec sheet GH200 Mini 576GB.pdfGH200 Mini 624GB: Spec sheet GH200 Mini 624GB.pdf
ManualsOfficial Nvidia GH200 Manual: https://docs.nvidia.com/grace/#grace-hopperOfficial Nvidia Grace Manual: https://docs.nvidia.com/grace/#grace-cpuOfficial Nvidia Grace getting started: https://docs.nvidia.com/grace/#getting-started-with-nvidia-graceGH200 576GB: Manual GH200 576GB.pdfGH200 624GB: Manual GH200 624GB.pdfGH200 Special Edition 576GB: Manual GH200 Special Edition 576GB.pdfGH200 Special Edition 624GB: Manual GH200 Special Edition 624GB.pdfGH200 Liquid 576GB: Manual GH200 Liquid 576GB.pdfGH200 Liquid 624GB: Manual GH200 Liquid 624GB.pdfGH200 Glass 576GB: Manual GH200 Glass 576GB.pdfGH200 Glass 624GB: Manual GH200 Glass 624GB.pdfGH200 Glass Special Edition 576GB: Manual GH200 Glass Special Edition 576GB.pdfGH200 Glass Special Edition 624GB: Manual GH200 Glass Special Edition 624GB.pdfGH200 Glass Liquid 576GB: Manual GH200 Glass Liquid 576GB.pdfGH200 Glass Liquid 624GB: Manual GH200 Glass Liquid 624GB.pdfGH200 Mini 576GB: Manual GH200 Mini 576GB.pdfGH200 Mini 624GB: Manual GH200 Mini 624GB.pdf
Operating systems:Ubuntu Server for ARM: https://ubuntu.com/download/server/armUbuntu Desktop for ARM: https://cdimage.ubuntu.com/daily-live/current/noble-desktop-arm64.iso
There are special Nvidia kernels for Ubuntu 22.04: https://packages.ubuntu.com/search?keywords=linux-nvidia-64k-hwe
Any other ARM linux distribution with kernel >6.2 should work just fine. Using the newest 64k kernel is highly recommended.
DriversNvidia GH200 drivers: https://www.nvidia.com/Download/index.aspx?lang=en-us
Select product type "data center", product series "HGX-Series" and operating system "Linux aarch64".PNY T400, Asus Dual GeForce RTX 4060 OC, Nvidia RTX A6000 Ada, Nvidia H100 drivers: https://www.nvidia.com/Download/index.aspx?lang=en-us
Select the corresponding product type, product series and operating system "Linux aarch64".Nvidia Bluefield-3 drivers: https://developer.nvidia.com/networking/doca#downloadsNvidia ConnectX-7 drivers: https://network.nvidia.com/products/ethernet-drivers/linux/mlnx_en/Intel E810-CQDA2 drivers: https://www.intel.com/content/www/us/en/download/19630/intel-network-adapter-driver-for-e810-series-devices-under-linux.html?wapkw=E810-CQDA2
Firmware/Bios:GH200 and GH200 Glass: Firmware Bios GH200 Q (Glass).zipGH200 Special Edition (Glass): coming soonGH200 Liquid (Glass): coming soonGH200 Mini: coming soonNvidia Bluefield-3: https://network.nvidia.com/support/firmware/bluefield3/Nvidia ConnectX-7: https://network.nvidia.com/support/firmware/connectx7/Intel E810-CQDA2: https://www.intel.com/content/www/us/en/search.html?ws=idsa-default#q=E810-CQDA2
SoftwareNvidia CUDA: https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=arm64-sbsaNvidia github: https://github.com/NVIDIANvidia container-toolkit: https://github.com/NVIDIA/nvidia-container-toolkitDocker: https://docs.docker.com/desktop/install/linux-install/Nvidia Tensorflow: https://github.com/NVIDIA/tensorflowNvidia Pytorch: https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorchHuggingface open source LLMs: https://huggingface.co/modelsList of inference servers: https://neptune.ai/blog/ml-model-serving-best-tools
White paper: Nvidia GH200 Grace-Hopper white paperNvidia Grace-Grace white paper
ZZZ UG (limited)
CEO: Bernhard Guentner
Trade register Bamberg HRB 10940
Try before you buy. You can apply for remote testing of a GH200 system. After approval, you will be given login credentials for remote access. If you want to come by and see it for yourself and run some tests, that is also possible any time.
Currently available for testing: GH200 576GB
Apply via email: email@example.com