Here’s why NVIDIA is paying $13 billion to buy Hugging Face, the company OpenAI agents hacked into last month 👀

Why is Nvidia's acquisition of Hugging Face bad news for Anthropic and Open AI? Well, to understand, we have to go deep into Open vs. Close Weight models. I'm going to try to break it all down for you in under three minutes. Nvidia's CEO, Jensen Wong, has called the AI ecosystem a five-layer cake, where Nvidia is the poster child for layer two, the chips, and Anthropic and Open AI are the poster children for layer four, the models. But did you know that Nvidia has models too? Although they're open source, which brings us to an important bifurcation, closed vs. open. We'll get to what that means in a second. To orient ourselves, let's talk about what companies have what models and whether they're open or closed. The frontier model companies mostly rely on closed weight models. In fact, Anthropic has never released an Open Weight model. The hyperscalers on the other hand have both. The Chinese companies mostly have Open Weight models, and Nvidia has not only never released a closed weight model, they're paying $13 billion to buy the company at the heart of Open Weight AI. To understand why, let's back up. To build into play a large language model or LLM, there are two distinct steps. The first is training, the lab stake text like the sentence the dog is sleeping, and split each word into tokens, which each get assigned a string of numbers called the vector. They then do math on it to try to relate each word to the most likely word that follows. To do that, the model has trillions of parameters, also called weights, which on a simpler scale we can think of like a coefficient when running a regression. For finance nerds, it's like calculating a stock's beta. Plot the return of say, meta, against the return of the S&P, and the slope of the line is your beta, or in this case our parameter or weight. Now in training began, these weights were all random noise, but over rounds it got nudged towards something meaningful. In fact, there was a 2013 study where they took the vector for king, subtracted the vector for man, and added women, and it lands in near the vector for queen. Alright, so after training, the final output is a file with billions of trillions of parameters which get frozen, and that brings us to the important second step, inference, which is where the open versus close weight comes into play. A close weight model means the model makers don't share their model, rather they keep it private and you can only use it by paying them or their license partners for access. But what's interesting, tying this back to video, is the chips needed to run inference, meaning for a user to ask the model a question and get an answer, don't have to be nearly as sophisticated as the chips will for training. In fact, many of the frontier companies and hyperscalers are now designing their own inference chips rather than relying on a video alone. And this brings us to the open weight ecosystem, where model makers allow anyone to download their file and run their model. Why would they do that? Well, let's plot model parameters from under a billion to 10 trillion against the memory needed to run the model. The more parameters, the more capable the model, but generally the slower it is and the more computed needs. Under about 100 gigabytes of memory, which typically supports around 100 billion parameters, you actually can run the model locally on your own computer. Over that, you typically need to rank compute from a cloud provider, which houses inference chips like Nvidia GPUs in data centers, which explains why companies like Google and Ellibaba give away models, they make money on compute, and why Nvidia needs the open weight ecosystem as well. Nvidia's biggest competitive threat is that as the largest customers are building their own inference chips, by owning hugging face, the site that everyone goes to for open weight models, Nvidia ensures customer diversification, and they can even make their deployment software, which runs on their chips, the path of least resistance.