Chinese businesses and universities turn to back alley GPU dealers to power AI

There's reportedly a bubbling back alley market for imported Nvidia graphics cards in [[link]] China. These are sought after chips globally, since the rise of large language models like ChatGPT, but in China there's a different challenge. That's because the US banned Nvidia from exporting its most powerful AI accelerators last year.

China is no longer allowed to directly import any Nvidia A100 or H100 GPUs. And before you think that's a chance for AMD to swoop in with [[link]] its massive Instinct cards, AMD's MI250 is also blocked from sale to the country.

China has to make do with less powerful options, or so it's decreed. Such options as AMD's MI100, which it said last September was not likely hit by the US sanctions, or Nvidia's purpose-designed A800 and H800. These two Nvidia designs are said to purposefully limit bandwidth to comply with the new export controls.

But that's clearly not enough for some. Reuters reports an underground chip market has popped up in light of the new restrictions on exports. 

These chip markets are largely said to be located in the city of Shenzhen, one of China's largest technology hubs. Vendors that Reuters spoke to say that they can provide small numbers of Nvidia A100 GPUs to customers willing to pay, let's say, over-the-odds for an AI accelerator. 

The report claims the cards are sold for around $20,000, which is roughly double the going price of $10,000. But similarly high demand for these chips due to a recent AI acceleration boom, one which is seeing Nvidia's Wall Street value top $1 trillion dollars, is also helping to keep prices high.

By comparison, double the price for a graphics card was altogether a pretty good deal during the crypto-mining craze.

Reuters spoke to 10 vendors that claimed they could "easily" secure small amounts of A100 cards, though these small-scale operations aren't likely enough to help China bolster its large language model and other large-scale AI operations. That's a large part of why the US has rolled out the new export controls on the country, as it fears these cards could also be used in a military context. Larger AI operations and training of massive models require masses of cards in the region of tens of thousands, depending on the task at hand.

Instead, these small sellers are likely a way for similarly smaller companies and startups to get hold of the computational power required to run their applications. Universities, local authorities, and AI startups are all noted as those that have sought out these back alley cards at some point or another.

One Hong Kong-based company was trying to purchase 2–4 A100 cards for their startup, though they note there was some risk involved in buying from these vendors. There's no whiff of a warranty here, nor customer support if something were to go wrong.

The US may end up changing its stance on how it limits the dealings of US companies, though it could still further bolster limits on the country. It holds that the current measures have already had a "substantial impact" on China's high-end GPU access. China, of course, disagrees with the sanctions altogether, noting that the US was trying to implement a "tech blockade" on the country back in September when the first GPU limitations were announced. 

Nvidia has said that it could lose money if China were altogether blocked from buying any of its cards. Though, of course, it has made it clear that it will follow the export controls tightly. 

In a statement to Reuters, Nvidia said, "If we receive information that a customer is breaching their agreement with us and exporting restricted products in violation of the law, we would take immediate and appropriate action."

This wouldn't be the first time that GPUs have been the subject of illicit dealings. China, or special administrative regions of China, has reported many cases of graphics cards being attempted to be smuggled into the country over the years, including 300 cards back in 2021 and 5,840 in 2022. However, this recent underground market does differ slightly. Rather than bypassing pesky taxation and import fees, it sounds like more legitimate businesses and users of cards are turning to [[link]] these vendors in an attempt to keep their AI work afloat.