The internet, once seen as a source of knowledge for all, has now become a complex maze. Although traditional search engines are powerful, they often inundate users with a flood of results, making it difficult to find what they are searching for. The emergence of new technologies like ChatGPT from OpenAI has been impressive, along with other language models such as Bard. However, these models also come with certain drawbacks for business users, such as the risk of generating inaccurate information, a lack of proper citation, potential copyright infringements, and a scarcity of reliable information in the business domain. The challenge lies not only in finding information but in finding the right information. In order to make Generative AI effective in the business world, we must address these concerns, which is the focal point of RAG.

The digital challenge: A sea of information

At the corner of platforms like Microsoft Copilot and Lucy is the transformative approach of the Retrieval-Augmented Generation (RAG) model.

Understanding RAG

What precisely is RAG, and how does it work? In simple terms, RAG is a two-step process:

1. Retrieval: Prior to providing an answer, the system delves into an extensive database, meticulously retrieving pertinent documents or passages. This isn’t a rudimentary matching of keywords; it’s a cutting-edge process that comprehends the intricate context and nuances of the query. RAG systems rely on the data owned or licensed by companies, and ensure that Enterprise Levels of access control are impeccably managed and preserved.

2. Generation: Once the pertinent information is retrieved, it serves as the foundation for generating a coherent and contextually accurate response. This isn’t just about regurgitating data; it’s about crafting a meaningful and informative answer.

By integrating these two critical processes, RAG ensures that the responses delivered are not only precise but also well-informed. It’s akin to having a dedicated team of researchers at your disposal, ready to delve into a vast library, select the most appropriate sources, and present you with a concise and informative summary.

Why RAG matters

Leading technology platforms that have embraced RAG – such as Microsoft Copilot for content creation or federated search platforms like Lucy – represent a significant breakthrough for several reasons:

1. Efficiency: Traditional models often demand substantial computational resources, particularly when dealing with extensive datasets. RAG, with its process segmentation, ensures efficiency, even when handling complex queries.

2. Accuracy: By first retrieving relevant data and then generating a response based on that data, RAG guarantees that the answers provided are firmly rooted in credible sources, enhancing accuracy and reliability.

3. Adaptability: RAG’s adaptability shines through as new information is continually added to the database. This ensures that the answers generated by platforms remain up-to-date and relevant.

RAG platforms in action

Picture yourself as a financial analyst seeking insights into market trends. Traditional research methods would require hours, if not days, to comb through reports, articles, and data sets. Lucy, however, simplifies the process – you merely pose your question. Behind the scenes, the RAG model springs into action, retrieving relevant financial documents and promptly generating a comprehensive response, all within seconds.

Similarly, envision a student conducting research on a historical event. Instead of becoming lost in a sea of search results, Lucy, powered by RAG, provides a concise, well-informed response, streamlining the research process and enhancing efficiency.

Take this one step further, Lucy feeds these answers across a complex data ecosystem to Microsoft Copilot and new presentations or documentation is created leveraging all of the institutional knowledge an organisation has created or purchased..

The road ahead

The potential applications of RAG are expansive, spanning academia, industry, and everyday inquiries. Beyond its immediate utility, RAG signifies a broader shift in our interaction with information. In an age of information overload, tools like Microsoft Copilot and Lucy, powered by RAG, are not merely conveniences; they are necessities.

Furthermore, as technology continues to evolve, we can anticipate even more sophisticated iterations of the RAG model, promising heightened accuracy, efficiency, and user experience. Working with platforms that have embraced RAG from the onset (or before even a term) will keep your organisation ahead of the curve.

Conclusion

In the digital era, we face both challenges and opportunities. While the sheer volume of information can be overwhelming, technologies like Microsoft Copilot or Lucy, underpinned by the potency of Retrieval-Augmented Generation, offer a promising path forward. This is a testament to technology’s potential not only to manage but also to meaningfully engage with the vast reservoirs of knowledge at our disposal. These aren’t just platforms; they are a glimpse into the future of information retrieval.

Photo by Markus Winkler on Unsplash

The post How information retrieval is being revolutionised with RAG technology appeared first on AI News.

The company first launched its XiaoIce project back in August 2017. In April, Microsoft said it had achieved full duplexing — the ability to speak and listen at the same time, similar to humans.

Microsoft’s announcement was made before Google’s demonstration earlier this month but, unlike Google, the company had nothing to show at the time.

XiaoIce has now been demonstrated in action during a London event:

The chatbot is only available in China at this time, but it’s become incredibly popular with more than 500 million users.

XiaoIce also features over 230 skills and has been used to perform things such as creating news and hosting radio programs as part of its ‘Content Creation Platform’.

In a blog post, Microsoft VP of AI Harry Shum revealed that more than 600,000 people have spoken on the phone with XiaoIce since it launched in August.

“Most intelligent agents today like Alexa or Siri focus on IQ or task completion, providing basic information like weather or traffic,” wrote Shum. “But we need agents and bots to balance the smarts of IQ with EQ – our emotional intelligence.”

“When we communicate, we use tone of voice, word play, and humour, things that are very difficult for computers to understand. However, Xiaoice has the ability to have human-like verbal conversations, which the industry calls full duplex.”

As many have called for since the Duplex demo, and Google has promised, Microsoft ensures a human participant is aware they’re speaking to an AI.

One thing we’d love to see is a conversation between XiaoIce and Google Duplex to see how well they each hold up. However, let’s keep our hands on the kill switch in case world domination becomes a topic.

What are your thoughts on conversational AIs like XiaoIce and Duplex? Let us know in the comments.

 Interested in hearing industry leaders discuss subjects like this and sharing their use-cases? Attend the co-located AI & Big Data Expo events with upcoming shows in Silicon Valley, London and Amsterdam to learn more. Co-located with the  IoT Tech Expo, Blockchain Expo and Cyber Security & Cloud Expo so you can explore the future of enterprise technology in one place.

The post Watch out Google Duplex, Microsoft also has a chatty AI appeared first on AI News.

Speaking at the Carnegie Mellon University – K&L Gates Conference on Ethics and AI in Pittsburgh, Eric Horvitz made the revelation. He says the group at Microsoft looking into possible misuse on a case-by-case basis is the Aether Committee (“Aether” stands for AI and Ethics in Engineering and Research.)

“Significant sales have been cut off,” Horvitz said. “And in other sales, various specific limitations were written down in terms of usage, including ‘may not use data-driven pattern recognition for use in face recognition or predictions of this type.’”

Horvitz, of course, did not reveal the specific companies of which Microsoft decided not to formulate a deal with. However, it’s pleasing to hear the company putting ethics above money when it comes to artificial intelligence. Abuses will be widely covered and hamper its potential.

Amidst the fallout of the Cambridge Analytica and Facebook scandal, and the use of this stolen data to target voters during the 2016 U.S. presidential campaign, people are naturally more wary of anything which involves mass data analysis.

Manipulating votes is one of the key concerns Horvitz raises for the abuse of AI, along with human rights violations, increasing the risk of physical harm, or preventing access to critical services and resources.

In reverse, we’ve already seen how AI itself can be manipulated — from Microsoft itself, no less. The company’s now infamous ‘Tay’ chatbot was taught by people online to spew racist comments. “It’s a great example of things going awry,” Horvitz acknowledged.

Rather than replace humans, Horvitz wants AI to be complementarity and not a replacement — often as more of a backstop for human decisions. However, it could still be used when invoked for tasks where a human would not be as effective.

For example, Horvitz highlights a program from Microsoft AI that helps caregivers to identify patients most at risk of being readmitted to a hospital within 30 days. Scholars who assessed the program determined that it could reduce rehospitalisations by 18 percent while cutting a hospital’s costs by nearly 4 percent.

The comments made by Horvitz once again highlight the need for AI companies to ensure their approach is responsible and ethical. The opportunities are endless if AI is developed properly, but it could just as easily lead to disaster if not.

Update: A previous headline ‘Microsoft has dropped some deals over AI ethical concerns’ was misconstrued as meaning the company dropped some existing deals. It has been updated to reflect Microsoft decided against some possible future partnerships over ethical concerns.

What are your thoughts on Microsoft’s approach to AI ethics? Let us know in the comments.

 Interested in hearing industry leaders discuss subjects like this and sharing their use-cases? Attend the co-located AI & Big Data Expo events with upcoming shows in Silicon Valley, London and Amsterdam to learn more. Co-located with the  IoT Tech Expo, Blockchain Expo and Cyber Security & Cloud Expo so you can explore the future of enterprise technology in one place.

The post Microsoft dropped some potential deals over AI ethical concerns appeared first on AI News.