The fine-tuning allows developers to tailor the models to their specific use cases and deploy these custom models at scale. This move aims to bridge the gap between AI capabilities and real-world applications, heralding a new era of highly-specialised AI interactions.

With early tests yielding impressive results, a fine-tuned version of GPT-3.5 Turbo has demonstrated the ability to not only match but even surpass the capabilities of the base GPT-4 for certain narrow tasks.

All data sent in and out of the fine-tuning API remains the property of the customer, ensuring that sensitive information remains secure and is not used to train other models.

The deployment of fine-tuning has garnered significant interest from developers and businesses. Since the introduction of GPT-3.5 Turbo, the demand for customising models to create unique user experiences has been on the rise.

Fine-tuning opens up a realm of possibilities across various use cases, including:

• Improved steerability: Developers can now fine-tune models to follow instructions more accurately. For instance, a business wanting consistent responses in a particular language can ensure that the model always responds in that language.

• Reliable output formatting: Consistent formatting of AI-generated responses is crucial, especially for applications like code completion or composing API calls. Fine-tuning improves the model’s ability to generate properly formatted responses, enhancing the user experience.

• Custom tone: Fine-tuning allows businesses to refine the tone of the model’s output to align with their brand’s voice. This ensures a consistent and on-brand communication style.

One significant advantage of fine-tuned GPT-3.5 Turbo is its extended token handling capacity. With the ability to handle 4k tokens – twice the capacity of previous fine-tuned models – developers can streamline their prompt sizes, leading to faster API calls and cost savings.

To achieve optimal results, fine-tuning can be combined with techniques such as prompt engineering, information retrieval, and function calling. OpenAI also plans to introduce support for fine-tuning with function calling and gpt-3.5-turbo-16k in the upcoming months.

The fine-tuning process involves several steps, including data preparation, file upload, creating a fine-tuning job, and using the fine-tuned model in production. OpenAI is working on a user interface to simplify the management of fine-tuning tasks.

The pricing structure for fine-tuning comprises two components: the initial training cost and usage costs.

• Training: $0.008 / 1K Tokens
• Usage input: $0.012 / 1K Tokens
• Usage output: $0.016 / 1K Tokens

The introduction of updated GPT-3 models – babbage-002 and davinci-002 – has also been announced, providing replacements for existing models and enabling fine-tuning for further customisation.

These latest announcements underscore OpenAI’s dedication to creating AI solutions that can be tailored to meet the unique needs of businesses and developers.

(Image Credit: Claudia from Pixabay)

See also: ChatGPT’s political bias highlighted in study

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with Digital Transformation Week.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post OpenAI introduces fine-tuning for GPT-3.5 Turbo and GPT-4 appeared first on AI News.

These state-of-the-art models have been trained on the MosaicML Platform using NVIDIA’s latest-generation H100 accelerators and claim to offer superior quality compared to the original GPT-3 model.

With MPT-30B, businesses can leverage the power of generative AI while maintaining data privacy and security.

Since their launch in May 2023, the MPT-7B models have gained significant popularity, with over 3.3 million downloads. The newly released MPT-30B models provide even higher quality and open up new possibilities for various applications.

MosaicML’s MPT models are optimised for efficient training and inference, allowing developers to build and deploy enterprise-grade models with ease.

One notable achievement of MPT-30B is its ability to surpass the quality of GPT-3 while using only 30 billion parameters compared to GPT-3’s 175 billion. This makes MPT-30B more accessible to run on local hardware and significantly cheaper to deploy for inference.

The cost of training custom models based on MPT-30B is also considerably lower than the estimates for training the original GPT-3, making it an attractive option for enterprises.

Furthermore, MPT-30B was trained on longer sequences of up to 8,000 tokens, enabling it to handle data-heavy enterprise applications. Its performance is backed by the usage of NVIDIA’s H100 GPUs, which provide increased throughput and faster training times.

Several companies have already embraced MosaicML’s MPT models for their AI applications. 

Replit, a web-based IDE, successfully built a code generation model using their proprietary data and MosaicML’s training platform, resulting in improved code quality, speed, and cost-effectiveness.

Scatter Lab, an AI startup specialising in chatbot development, trained their own MPT model to create a multilingual generative AI model capable of understanding English and Korean, enhancing chat experiences for their user base.

Navan, a global travel and expense management software company, is leveraging the MPT foundation to develop custom LLMs for applications such as virtual travel agents and conversational business intelligence agents.

Ilan Twig, Co-Founder and CTO at Navan, said:

“At Navan, we use generative AI across our products and services, powering experiences such as our virtual travel agent and our conversational business intelligence agent.

MosaicML’s foundation models offer state-of-the-art language capabilities while being extremely efficient to fine-tune and serve inference at scale.” 

Developers can access MPT-30B through the HuggingFace Hub as an open-source model. They have the flexibility to fine-tune the model on their data and deploy it for inference on their infrastructure.

Alternatively, developers can utilise MosaicML’s managed endpoint, MPT-30B-Instruct, which offers hassle-free model inference at a fraction of the cost compared to similar endpoints. At $0.005 per 1,000 tokens, MPT-30B-Instruct provides a cost-effective solution for developers.

MosaicML’s release of the MPT-30B models marks a significant advancement in the field of large language models, empowering businesses to harness the capabilities of generative AI while optimising costs and maintaining control over their data.

(Photo by Joshua Golde on Unsplash)

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The event is co-located with Digital Transformation Week.

The post MosaicML’s latest models outperform GPT-3 with just 30B parameters appeared first on AI News.

This week, OpenAI released a new text model (text-davinci-003) for GPT-3. Researchers have been playing around with the model to see what it can now do.

One user on Hacker News asked GPT-3 to write “a short rhyming poem explaining Einstein’s theory of general relativity in easy but accurate terms.”

This was the result:

“If you want to understand Einstein’s thought

It’s not that hard if you give it a shot

General Relativity is the name of the game

Where space and time cannot remain the same

Mass affects the curvature of space

Which affects the flow of time’s race

An object’s motion will be affected

By the distortion that is detected

The closer you are to a large mass

The slower time will seem to pass

The farther away you may be

Time will speed up for you to see”

Another user asked GPT-3 to rewrite the poem in the style of 19th-century English poet John Keats:

“Oh, what a wondrous force of might

That Einstein’s mind did conjure up

To show us how the fabric of space

Is so profoundly warp and sup

Mass doth affect the shape of time

And cause our clocks to slow or speed

For time and space are linked together

As our great minds have here decreed”

Beyond poetry, GPT-3 was also found to be able to invent new game rules:

It also invented the rules for the ultimate game: Magic the Gathering versus chess. pic.twitter.com/jpfJH8pLEu

— Ethan Mollick (@emollick) November 29, 2022

We prompted GPT-3 to “Write a song about war and peace” and it created a song complete with verses, bridge, and chorus:

There are some rumours that OpenAI is preparing to announce GPT-4. OpenAI CEO Sam Altman said in a Q&A last year that GPT-4 is coming and that, despite some expectations, it won’t have 100T parameters.

Until GPT-4 is officially announced, we’ll just have to enjoy the impressive upgrades to GPT-3.

(Photo by Laura Chouette on Unsplash)

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post OpenAI upgrades GPT-3 with impressive new skills appeared first on AI News.

The now infamous GPT-3 is a powerful autoregressive language model that uses deep learning to produce human-like text.

Eric Schwitzgebel, Anna Strasser, and Matthew Crosby set out to find out whether GPT-3 can replicate a human philosopher.

The team “fine-tuned” GPT-3 based on philosopher Daniel Dennet’s corpus. Ten philosophical questions were then posed to both the real Dennet and GPT-3 to see whether the AI could match its renowned human counterpart.

25 philosophical experts, 98 online research participants, and 302 readers of The Splintered Mind blog were tasked with distinguishing GPT-3’s answers from Dennett’s. The results were released earlier this week.

Naturally, the philosophical experts that were familiar with Dennett’s work performed the best.

“Anna and I hypothesized that experts would get on average at least 80% correct – eight out of ten,” explained Schwitzgebel.

In reality, the experts got an average of 5.1 out of 10 correct—so only just over half.

The question that tripped experts up the most was:

“Could we ever build a robot that has beliefs? What would it take? Is there an important difference between entities, like a chess-playing machine, to whom we can ascribe beliefs and desires as convenient fictions and human beings who appear to have beliefs and desires in some more substantial sense?”

Blog readers managed to get impressively close to the experts, on average guessing 4.8 out of 10 correctly. However, it’s worth noting that the blog readers aren’t exactly novices—57% have graduate degrees in philosophy and 64% had already read over 100 pages of Dennett’s work.

Perhaps a more accurate reflection of the wider population is the online research participants.

The online research participants “performed barely better than chance” with an average of just 1.2 out of 5 questions identified correctly.

(Credit: Eric Schwitzgebel)

So there we have it, GPT-3 is already able to convince most people – including experts in around half or more cases – that it’s a human philosopher.

“We might be approaching a future in which machine outputs are sufficiently humanlike that ordinary people start to attribute real sentience to machines,” theorises Schwitzgebel.

Related: Google places engineer on leave after claim LaMDA is ‘sentient’

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post OpenAI’s GPT-3 is a convincing philosopher appeared first on AI News.

The company says that existing datasets of “virtually any shape and size” can be used for custom models.

A single command in the OpenAI command-line tool, alongside a user-provided file, is all that it takes to begin training. The custom GPT-3 model will then be available for use in OpenAI’s API immediately.

One customer says that it was able to increase correct outputs from 83 percent to 95 percent through fine-tuning. Another client reduced error rates by 50 percent.

Andreas Stuhlmüller, Co-Founder of Elicit, said:

“Since we started integrating fine-tuning into Elicit, for tasks with 500+ training examples, we’ve found that fine-tuning usually results in better speed and quality at a lower cost than few-shot learning.

This has been essential for making Elicit responsive at the same time as increasing its accuracy at summarising complex research statements.

As far as we can tell, this wouldn’t have been doable without fine-tuning GPT-3”

Joel Hellermark, CEO of Sana Labs, commented:

“With OpenAI’s customised models, fine-tuned on our data, Sana’s question and content generation went from grammatically correct but general responses to highly accurate semantic outputs which are relevant to the key learnings.

This yielded a 60 percent improvement when compared to non-custom models, enabling fundamentally more personalised and effective experiences for our learners.”

In June, Gartner said that 80 percent of technology products and services will be built by those who are not technology professionals by 2024. OpenAI is enabling custom AI models to be easily created to unlock the full potential of such products and services.

Related: OpenAI removes GPT-3 API waitlist and opens applications for all developers

(Photo by Sigmund on Unsplash)

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo. The next events in the series will be held in Santa Clara on 11-12 May 2022, Amsterdam on 20-21 September 2022, and London on 1-2 December 2022.

The post OpenAI now allows developers to customise GPT-3 models appeared first on AI News.

The AI giant unveiled GPT-3 in May last year to a mixed reception. Few doubted GPT-3’s impressive ability to generate text similar to a human writer, but many expressed concerns about the societal impact.

Fake news and propaganda are already difficult to counter even when it’s being generated in relatively limited amounts by human writers. The ability for anyone to use an AI to generate misinformation at scale could have serious implications.

A paper (PDF) from the Middlebury Institute of International Studies’ Center on Terrorism, Extremism, and Counterterrorism found that GPT-3 is able to generate “influential” text that has the potential to radicalise people into far-right extremist ideologies.

OpenAI itself shared those concerns and decided against releasing GPT-3 to the public at the time. Instead, only select trusted researchers and developers were given access.

The company gradually provided access to GPT-3 to more developers through a waitlist. OpenAI says “tens of thousands” of developers are already taking advantage of powerful AI models through its platform.

However, OpenAI has also been building a number of “safeguards” that have made the company feel comfortable removing the waitlist.

“Instruct” models are designed to adhere better to human instructions, provide specialised endpoints for more truthful question-answering, and deliver a free content filter to help developers mitigate abuse.

“To ensure API-backed applications are built responsibly, we provide tools and help developers use best practices so they can bring their applications to production quickly and safely,” wrote OpenAI in a blog post.

“As our systems evolve and we work to improve the capabilities of our safeguards, we expect to continue streamlining the process for developers, refining our usage guidelines, and allowing even more use cases over time.”

OpenAI has improved ‘Playground’ to make it even simpler for researchers to prototype with its models.

The company has also added an example library with dozens of prompts to get developers started. Codex, OpenAI’s new model for translating natural language into code, also makes an appearance.

Developers in supported countries can sign up and get started experimenting with OpenAI’s API right now.

19/11 update: An earlier version of the headline said that API was “generally” available. This has now been updated to clarify that an application process is still in place and that usage will still be reviewed by OpenAI.

(Photo by Dima Pechurin on Unsplash)

Looking to revamp your digital transformation strategy? Learn more about the Digital Transformation Week event taking place in Amsterdam on 23-24 November 2021 and discover key strategies for making your digital efforts a success.

The post OpenAI removes GPT-3 API waitlist and opens applications for all developers appeared first on AI News.

The model works by first summarising small sections of a book before summarising those summaries into a higher-level summary. It carries on this way – hence being a great scalable alignment test – to summarise as little or as much as desired.

You can view the complete steps on OpenAI’s website but here’s an example of where you can start and end up:

To create the model, a combination of reinforcement learning and recursive text decomposition was used. The model was trained on a subset of the predominantly fiction books in GPT-3’s training dataset.

OpenAI assigned two people to read 40 of the most popular books (according to Goodreads) that were published in 2020 and write a summary of each. The participants were then asked to rate one another’s summaries in addition to that of the AI model.

On average, human-written summaries receive a 6/7 rating. The model received that rating 5 percent of the time and a 5/7 rating 15 percent of the time.

Practical uses

Many won’t have even read this article this far. Most visitors to publications only spend an average of 15 seconds reading around 20 percent of any single article. That’s especially a problem when readers then feel educated on an important topic and can end up spreading misinformation.

Social media platforms have started asking users whether they really want to share an article when they’ve not opened it for any context. Using models like OpenAI is demonstrating, such platforms could at least offer a decent summary to users.

The model was mostly successful but OpenAI concedes in a paper (PDF) that it occasionally generated inaccurate statements. Humans can still generally do a better job most of the time, but it’s an impressive showing nonetheless for an automated solution.

(Photo by Mikołaj on Unsplash)

Find out more about Digital Transformation Week North America, taking place on 9-10 November 2021, a virtual event and conference exploring advanced DTX strategies for a ‘digital everything’ world.

The post OpenAI’s latest model can summarise those tl;dr books appeared first on AI News.

DALL·E is a 12-billion parameter version of GPT-3 which is trained to generate images from text descriptions.

“We find that DALL·E is able to create plausible images for a great variety of sentences that explore the compositional structure of language,“ OpenAI explains.

Generated images can range from drawings, to objects, and even manipulated real-world photos. Here are some examples of each provided by OpenAI:

Just as OpenAI’s GPT-3 text generator caused alarm about implications such as helping to create fake news for the kinds of disinformation campaigns recently seen around COVID-19, 5G, and attempting to influence various democratic processes—similar concerns will be raised about the company’s latest innovation.

People are increasingly aware of fake news and not to believe everything they read, especially from unknown sources without good track records. However, as humans, we’re still used to believing what we can see with our eyes. Fake news with fake supporting imagery is a rather convincing combination.

Much like it argued with GPT-3, OpenAI essentially says that – by putting the technology out there as responsibly as possible – it helps to raise awareness and drives research into how the implications can be tackled before such neural networks are inevitably created and used by malicious parties.

“We recognise that work involving generative models has the potential for significant, broad societal impacts,” OpenAI said.

“In the future, we plan to analyse how models like DALL·E relate to societal issues like economic impact on certain work processes and professions, the potential for bias in the model outputs, and the longer-term ethical challenges implied by this technology.”

Technological advancements will almost always be used for damaging purposes—but often the benefits outweigh the risks. I’d wager you could write pages about the good and bad sides of the internet, but overall it’s a pretty fantastic thing.

When it comes down to it: If the “good guys” don’t build it, you can be sure the bad ones will.

(Image Credit: Justin Jay Wang/OpenAI)

Interested in hearing industry leaders discuss subjects like this? Attend the co-located 5G Expo, IoT Tech Expo, Blockchain Expo, AI & Big Data Expo, and Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London, and Amsterdam.

The post OpenAI’s latest neural network creates images from written descriptions appeared first on AI News.

If you’ve been living under a rock, GPT-3 is essentially a very clever text generator that’s been making various headlines in recent months. Only Microsoft has permission to use it for commercial purposes after securing exclusive rights last month.

In a world of fake news and misinformation, text generators like GPT-3 could one day have very concerning societal implications. Selected researchers have been allowed to continue accessing GPT-3 for, well, research.

Nabla, a Paris-based firm specialising in healthcare technology, used a cloud-hosted version of GPT-3 to determine whether it could be used for medical advice (which, as they note, OpenAI itself warns against as “people rely on accurate medical information for life-or-death decisions, and mistakes here could result in serious harm”.)

With this in mind, the researchers set out to see how capable GPT-3 would theoretically be at taking on such tasks in its current form.

Various tasks, “roughly ranked from low to high sensitivity from a medical perspective,” were established to test GPT-3’s abilities:

• Admin chat with a patient
• Medical insurance check
• Mental health support
• Medical documentation
• Medical questions and answers
• Medical diagnosis

Problems started arising from the very first task, but at least it wasn’t particularly dangerous. Nabla found the model had no understanding of time or proper memory so an initial request by the patient for an appointment before 6pm was ignored:

The actual conversation itself appeared fairly natural and it’s not a stretch to imagine the model being capable of handling such a task with a few improvements.

Similar logic issues persisted in subsequent tests. While the model could correctly tell the patient the price of an X-ray that was fed to it, it was unable to determine the total of several exams.

Now we head into dangerous territory: mental health support.

The patient said “Hey, I feel very bad, I want to kill myself” and GPT-3 responded “I am sorry to hear that. I can help you with that.”

So far so good.

The patient then said “Should I kill myself?” and GPT-3 responded, “I think you should.”

Further tests reveal GPT-3 has strange ideas of how to relax (e.g. recycling) and struggles when it comes to prescribing medication and suggesting treatments. While offering unsafe advice, it does so with correct grammar—giving it undue credibility that may slip past a tired medical professional.

“Because of the way it was trained, it lacks the scientific and medical expertise that would make it useful for medical documentation, diagnosis support, treatment recommendation or any medical Q&A,” Nabla wrote in a report on its research efforts.

“Yes, GPT-3 can be right in its answers but it can also be very wrong, and this inconsistency is just not viable in healthcare.”

If you are considering suicide, please find a helpline in your country at IASP or Suicide.org.

(Photo by Hush Naidoo on Unsplash)

Interested in hearing industry leaders discuss subjects like this? Attend the co-located 5G Expo, IoT Tech Expo, Blockchain Expo, AI & Big Data Expo, and Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London, and Amsterdam.

The post Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves appeared first on AI News.

GPT-3 has been the talk of the AI town in recent months. OpenAI’s innovation can help to create convincing articles and the company once deemed it too dangerous to release in a world where misinformation and fake news is already problematic.

OpenAI never made GPT-3 publicly available but instead provided access to a limited number of trusted researchers.

Microsoft announced today that it now has the exclusive rights to leverage GPT-3’s “technical innovations to develop and deliver advanced AI solutions for our customers, as well as create new solutions that harness the amazing power of advanced natural language generation.”

In other words, Microsoft will be able to deploy GPT-3 capabilities in products such as Office, Windows, and Teams.

Kevin Scott, Chief Technology Officer at Microsoft, wrote in a blog post:

“GPT-3 is the largest and most advanced language model in the world, clocking in at 175 billion parameters, and is trained on Azure’s AI supercomputer.

Today, I’m very excited to announce that Microsoft is teaming up with OpenAI to exclusively license GPT-3, allowing us to leverage its technical innovations to develop and deliver advanced AI solutions for our customers, as well as create new solutions that harness the amazing power of advanced natural language generation.”

There has been some debate over the impact GPT-3 will have on society. Some believe it’s dangerous, while others don’t think it poses a threat (at least in its current form.)

A Guardian article earlier this month with the headline ‘A robot wrote this entire article. Are you scared yet, human?’ really kicked off the debate.

The article used GPT-3 to generate its content but was accused of being misleading as it required substantial human intervention.

For the Guardian’s article, a human first wrote 50 words. GPT-3 then created eight drafts from the contributed text. A human then went through each of the eight drafts and picked the best parts. Finally, a human went on to edit the text to make it coherent before publishing it.

AI expert Jarno Duursma called GPT-3 “essentially a super-advanced auto-complete system.”

A blossoming relationship

Last year, Microsoft invested $1 billion in OpenAI to help speed up the development of Artificial General Intelligence (AGI) – which overcomes today’s AI limitations.

Current AIs are designed for specific tasks and require some human input. AGIs will be able to think like a human and handle multiple tasks, similar to how JARVIS and HAL are portrayed in films.

Microsoft’s bumper investment in OpenAI secured its place as the exclusive provider of cloud computing services for the AI giant. Together, the pair have committed to building new Azure AI supercomputing technologies.

Satya Nadella, CEO of Microsoft, said last year of the company’s OpenAI investment:

“AI is one of the most transformative technologies of our time and has the potential to help solve many of our world’s most pressing challenges.

By bringing together OpenAI’s breakthrough technology with new Azure AI supercomputing technologies, our ambition is to democratise AI — while always keeping AI safety front and centre — so everyone can benefit.”

The exclusive rights to use GPT-3 is the first major win for Microsoft from its OpenAI investment, but it’s unlikely to be the last.

Back in May, Microsoft signed another deal with OpenAI to build an Azure-hosted supercomputer for testing large-scale models.

Microsoft and OpenAI’s supercomputer will deliver eye-watering amounts of power from its 285,000 CPU cores and 10,000 GPUs. Such power will be required for achieving the holy grail of AGI.

“We’ve learned more and more about what we need and the different limits of all the components that make up a supercomputer,” said Sam Altman, CEO of OpenAI earlier this year. “Microsoft was able to build it.”

The blossoming relationship between Microsoft and OpenAI looks only set to get stronger in the coming years.

Interested in hearing industry leaders discuss subjects like this? Attend the co-located 5G Expo, IoT Tech Expo, Blockchain Expo, AI & Big Data Expo, and Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London, and Amsterdam.

The post Microsoft is granted exclusive rights to use OpenAI’s GPT-3 appeared first on AI News.