The revolution raised by ChatGPT does not come cheap: a few days ago we learned that Microsoft invested hundreds of millions of dollars to buy NVIDIA A100 graphics cards so that OpenAI could train its model and then integrate it into Bing. It therefore seemed that only large companies could agree to create their own ChatGPT, but things have changed.
NVIDIA DGX Cloud. This company has just announced this service that allows you to rent virtual versions of its DGX servers, each of which has eight NVIDIA H100 or A100 and 640 GB of memory. If the client needs it, the service can scale up to 32,000 GPUs, but it’s almost better not to think about the cost of that option because the basic one with those eight high-performance graphics cards is very, very expensive. NVIDIA has a potential reef here.
35,000 euros per month. More or less that ($36,999) is what a month’s rental of the basic service costs with the NVIDIA A100. This option, they say at NVIDIA, will allow the customer who opts for it to have “full-time reserved access” to those GPUs that they rent and that they will not have to share with anyone. A DGX server is around $200,000, for example.
Training models costs a fortune. The investment that Microsoft made to train Bing with ChatGPT and have it “ingest” huge amounts of data and then “learn” from it is enormous, but it was already known that this type of process has a very high cost. Last year Google told us about its PaLM model, with 540,000 million parameters (the more, the more accurate and better the model behaves), and according to some estimates it cost between 9 and 23 million dollars to train it. Meta’s LLaMA model, for example, was trained for 21 days on 2,048 NVIDIA A100 GPUs, which cast a cost about four million dollars. Other studies show similar figures to model them with about 175,000 million parameters.
Gradually options emerge. However, the NVIDIA service gives access to anyone with sufficient resources —without having to be a Microsoft— to create and train their own ChatGPT. MosaicML, a company specialized in this field, offered months ago a process to create a chatbot with the quality of GPT-3 for less than $500,000.
sow to reap. This enormous investment in the training of these models is what logically now means that companies like OpenAI, which were born with an open and non-profit philosophy, have completely changed to a business model with a lot of secrecy and in which the income is probably will grow significantly, especially with the launch of GPT-4 and its API. Microsoft, which announced its Copilot for its Microsoft 365 platform, will offer it as a high-cost side for companies that want to take advantage of these capabilities in the applications and services of that suite.
In Xataka: I have no idea about programming but thanks to GPT-4 I have created a clone of Flappy Bird
