open-source chatbot launched in Silicon Valley

A San Francisco-based startup last valued at $38 billion, on Friday released open-source code that companies could use to create their own chatbots, along the lines of OpenAI's ChatGPT.

The code is an AI model, an algorithm that can learn from new data to perform a variety of tasks, such as a set of data.

Databricks CEO Ali Ghodsi said the release was intended to demonstrate a viable alternative to training a kind of AI model called a large language model with enormous resources and computing power.

A large language model is used to build OpenAI's viral chatbot, ChatGPT. OpenAI, valued at $29 billion, trains its AI models with huge troves of data on a supercomputer from the investor Microsoft Corp. The computing costs are eye-watering according to OpenAI CEO Sam Altman.

The OpenAI charges business for access to its models for their own applications and has projected $1 billion in sales by the year 2024.

Databricks' effort comes with caveats. Ghodsi said that while the open-source chatbot displayed impressive capabilities at such tasks as writing blog posts, the company had not released formal benchmark tests to show that the bot matched ChatGPT's performance.

Databricks sells cloud-based data mining and analytics software to businesses and said last year it had surpassed $1 billion in annualized revenue.

Databricks wants enterprises to train their AI models using its software. Ghodsi said that the company's researchers had taken a two-year old model that was freely available and trained it with a small amount of data for three hours on a single computer that anyone with a credit card could rent.

The future will be that everyone has their own model, and they can actually train it, and they can make it better, Ghodsi said. That's why they don't have to give their data to someone else. Databricks' move comes at a time when startups are raising millions of dollars of venture capital investment to train their AI models and tech companies such as Alphabet's Google and Meta platforms are trying to shrink the size and cost of AI models while improving their accuracy.

In the end, my belief is that you will make these models smaller, smaller and smaller, and they will be open-sourced, said Ghodsi.