The more AI features an app uses, the greater the expense for the company building it, squeezing profit margins. But some companies have already found ways to control costs.

Microsoft (ASX: MSFT)-owned LinkedIn said that although usage of its AI recruiting assistant tool has continued to climb over the past year, the cost of running the tool has remained steady. The tool relies on large language models to help recruiters automatically screen candidates and reach out to them.

LinkedIn revealed late last month that more than 25,000 companies are paying to use the AI recruiting assistant, up from 20,000 disclosed in September. LinkedIn executives said in interviews that the customer count has grown steadily over the past year, but the cost of the product and service has barely changed. LinkedIn Chief Technology Officer Erran Berger said that although the user base for AI features is expected to keep expanding, the company does not plan to increase its investment in graphics processing units (GPUs) during the current fiscal year ending in June 2027.

"We can build increasingly powerful tools while keeping costs within a reasonable range," he said. Such cost-control capability is crucial for Microsoft. Like other major cloud service providers, Microsoft is expanding data center capital expenditure, which erodes the company's free cash flow, and the company faces enormous pressure to cut spending.

To achieve this, LinkedIn no longer relies entirely on OpenAI's general-purpose large models, instead developing its own lightweight AI models through knowledge distillation. Knowledge distillation refers to customized training of small models so that they mimic the output quality of large models in limited task scenarios. Vice President of Engineering Prashanth Padmanabhan explained that LinkedIn's approach is to use OpenAI models to "teach" small open-source models to master recruitment-related automation tasks. (LinkedIn declined to disclose which specific open-source models it uses.)

Padmanabhan said that when the AI recruiting assistant was first launched a year ago, the vast majority of functions relied on OpenAI models; now, tasks such as candidate search, ranking candidates according to recruitment requirements, and drafting candidate communication messages are all handled by lightweight models. The output quality of the small models is close to that of OpenAI models, and they can save up to 67% of operating costs compared with directly calling OpenAI a year ago. (Since then, OpenAI has lowered prices for multiple models and launched more efficient new versions; LinkedIn did not say whether its self-developed small models still have a cost advantage over OpenAI's current versions.)

However, Padmanabhan also noted that LinkedIn's newly launched AI features still first use OpenAI models to provide capabilities, and are replaced only after customized small models are developed. This also explains why overall computing costs have only remained stable rather than falling further. "We still use frontier large models in many business scenarios, but through practice, we have found that relying on frontier large models is not the optimal solution for scaling," Padmanabhan said. "We want to balance product innovation and engineering efficiency, so that even as product scale continues to grow, end-to-end overall costs remain stable."

If other companies follow LinkedIn's approach, they could even achieve greater cost savings. Thanks to Microsoft's long-term commercial agreement with OpenAI, LinkedIn and companies within the Microsoft ecosystem do not need to pay model licensing fees to call OpenAI models, only bearing the computing power operating costs. In other words, all of LinkedIn's savings come from the reduction in computing power brought by lightweight models. For external companies that need to pay full price to call OpenAI and Anthropic APIs, switching to third-party small models could deliver even more significant cost reductions.

Of course, LinkedIn has an advantage that other AI application developers find hard to match: the agreement between the two sides allows LinkedIn to perform knowledge distillation on OpenAI's models, while OpenAI prohibits other companies from distilling its models to develop competing products. Even so, companies such as AT&T and Cursor have already achieved good results by customizing open-source models without distillation technology.