The rtx 4090 has 512 tensor cores and the h100 has 456 tensor cores. I thought that tensor cores were much more important for AI and ML workloads.

Why does the rtx 4090 have more tensor cores than the h100? h100 being a data center gpu? I would expect it to have more tensor cores than the rtx 4090. Can someone explain this to me?

  • You're right that tensor core count is important to AI/ML performance, but there's more to consider here. Memory bandwidth, and to a lesser extent capacity, is hugely important, and the H100 dedicates a significant amount of area to its 80 GB of HBM. That (along with a huge amount of cache), takes up space on the interposer, and neither die area nor interposer area are free.
    – JMY1000
    Commented Apr 15 at 5:51
  • If you want to learn more, this post may be a good place to start. That said, since this isn't actually asking for a particular recommendation, I'm going to consider it off topic and close it for now. If you decide you actually want to try and buy some H100s (god forbid) and need recommendations, go ahead and edit your question to reflect that and we can reopen it.
    – JMY1000
    Commented Apr 15 at 5:52


Browse other questions tagged or ask your own question.