To develop artificial intelligence computation power, we need several factors—one of them is access to a special type of chip called Graphics Processing Units, or GPUs. Creating GPU chips is a highly skilled operation and involves huge capital expenditures. Given these special requirements, only a few companies in the world have the capability to make GPUs.Globally, there is a
shortage of AI computing prowess due to high demand and a low supply of GPU chips. Cut to home, the government of India has been working with the industry to work out investment proposals and measures to provide the necessary GPU infrastructure as a public resource.In the past, there has been no dedicated allocation of funds in Union budgets for computation infrastructure. Given the past developments around generative AI and the increasing demand to strengthen India’s position in the global AI map, the Indian AI community would have expected good news on February 1.However, given that this February will be an interim budget, no major announcements will follow. This means only once the newly elected government presents the full budget later this year will we get to know whether the government wants to allocate funds to build the narrative of Indian AI computation power.This post attempts to briefly explain the concept of GPUs, why they are important, and what India broadly needs (from immediate to mid-term) in terms of GPU power.
What are GPUs?Typically, a
generative AI framework can be broken down into four distinct layers: Infrastructure, platform, model, and application/services. The infrastructure
layer can be further broken down into sub-layers or components, which include both hardware and software. These typically require:
- GPUs
- High-performance computing (like a line of servers and workstations)
- Software to manage the workload across GPUs.
- High-speed, low-latency networking protocol.
At the infrastructure layer, it is critical to understand that both CPU and GPU are essential, silicon-based microprocessors, but GPUs
allocate a higher proportion of transistors to arithmetic logic units (ALUs) and fewer to caches and flow control mechanisms compared to CPUs. In simpler terms, GPUs represent a class of highly specialised silicon chips meticulously fine-tuned to excel in processing AI workloads. The primary difference between a CPU and GPU (both are silicon-based microprocessor units) is that a CPU handles all the main functions of a computer, whereas the GPU is a specialised component that excels at running many smaller tasks at once.This
blog nicely explains that CPU is like a head chef in a large restaurant who must make sure hundreds of burgers get flipped. Even if the head chef can do it personally, it’s not the best use of time. All kitchen operations may halt or slow down while the head chef is completing this simple but time-consuming task. To avoid this, the head chef can use junior assistants who flip several burgers in parallel. The GPU is more like a junior assistant with ten hands who can flip 100 burgers in 10 seconds.CPUs are notorious for their power-hungry nature, leading to elevated operational expenses and environmental repercussions stemming from increased power consumption. In stark contrast, AI chips are meticulously crafted with energy efficiency in mind, curbing power usage while maintaining robust performance. This distinctive feature positions them as a sustainable alternative for high-performance computing needs.
Traditionally, data centres have relied on CPUs to perform general-purpose computations, like data storage, indexation, and retrieval.However, for any applications of Gen AI, the requirement is for higher and parallel computing power, which can only be provided by GPUs. ChatGPT, for example, reportedly used 10,000
NVIDIA GPUs to train the model.
Global trend on AI expenditureWith the rise of generative AI, the global trend indicates an increasing need for investment in AI infrastructure, including GPUs, to train, run, and deploy AI models across the interconnected ecosystems of hardware, software, and data. The global AI market size was estimated at USD 136.55 billion in 2022. A
survey of 19 industries in 32 countries suggests that AI spending is expected to reach 154 USD bn in FY 23.In the US, the
National AI Research Resource (NAIRR) was first conceptualised under the National AI Initiative Act of 2022. The NAIRR Task Force
estimated that a budget of $2.6 billion would be required to set up the NAIRR over an initial six-year period. In July 2022, the U.S. Congress
passed the CHIPS and Science Act of 2022, which includes a package of 52 billion USD to boost US domestic semiconductor manufacturing.Last year, the UK government
committed £1 billion for the next generation of supercomputing and AI research. Again, in November 2023, the
UK government decided to invest £225 million (around $278 million) to set up a research facility for AI at the University of Bristol. As per the announcement, Hewlett-Packard Enterprise is going to build and deliver a new supercomputer system with over 5000 NVIDIA GH200 superchips.
What does India need?Recently, the Minister of State, MeitY, Shri Rajeev Chandrasekhar, indicated that the Government of India is contemplating setting up a GPU cluster under the India AI program. At present, India does not manufacture GPU chips. Our semi-conductor industry is at a nascent stage. This means that while India is building its capacity to manufacture AI chips, for now, we need to import GPU chips.As far as the existing compute capacity is concerned, India's existing high-end GPU compute infrastructure, e.g., AIRAWAT and PARAM Siddhi-AI, developed by CDAC, currently consists of only 656 GPUs. Whereas, in
comparison, the world’s fastest supercomputers have more than 30,000 GPUs. As per estimates and the recent joint
report of Nasscom and Deloitte, India needs to have access to GPU infrastructure with at least
exaflop AI capacity and 25,000 high-performance H100 GPUs. This compute estimate is in sync with the latest recommendation of
Expert Working Group 6 on Future Labs Compute, which was constituted by the Ministry of Electronics and Information Technology (MeitY).India can achieve this target either through setting up its own infrastructure and/or in combination with buying access from GPU cloud providers and system integrators. Tech companies typically buy access to AI chips and their compute power through cloud computing services. That way, they do not have to build and operate their own data centres full of computer servers connected with specialised networking gear.Globally, there is a shortage of GPUs; for instance, start-ups are heavily dependent on GPU clouds and are struggling to get sufficient hourly credits to access GPUs. Besides, the market is getting monopolised. Therefore, to design a sovereign and sustainable solution, a beginning towards making necessary investments (at this stage, this means buying GPU chips) at the infrastructure layer is critical and can have the following benefits:
- First, if the government provides the requisite infrastructure layer, it could benefit the other layers and the downstream ecosystem. For instance, start-ups could build or train models using the infrastructure layer; eventually, this will spur activities at the application layer and gradually add to the deployment and adoption rate of AI across sectors.
- Second, this could ensure India’s long-term competitiveness and leadership in AI, especially in building foundational models and Gen AI models as digital public goods, like native large language models.
- Third, India has a rich repository of diverse data (unlike most countries) generated through the existing digital public infrastructure (DPIs). The joint report of Nasscom and Deloitte recommends that integrating Gen AI into our existing DPI (like Aadhaar and UPI) could enhance its technological capabilities, inclusive access, and innovation across sectors.
However, GPUs cost a lot of money. As per industry estimates, the total cost of the 25000 A100 GPU cluster will be approximately Rs 12,484 Cr (1.5 billion USD), whereas it will cost approximately Rs 22,472 Cr (2.7 billion USD) for the 25000 H100 40 GB GPU. These are the estimated costs of two different categories of GPUs supplied by NVIDIA, i.e.,
A 100 40 GB GPU and
H 100 40 GB GPU (H 100 is the most advanced version of GPU).Some experts have forecast that while 25000 GPUs is a good starting point, it is insufficient to compete globally. Irrespective of whether the prediction comes true, we need to prepare in advance to figure out alternative sources of investment to keep building computation power. Perhaps public-private partnerships and repurposing and reprioritizing existing public expenditures could be some of the options. If India wants to stay relevant in the AI world, it must spend.
Sudipto Banerjee is an independent legal consultant. Views expressed in the above piece are personal and solely that of the author. They do not necessarily reflect Firstpost’s views.Read all the Latest News, Trending News, Cricket News, Bollywood News,India News and Entertainment News here. Follow us on Facebook, Twitter and Instagram.
