Delivering this without artificial intelligence (AI) would be comparable to traversing the Amazon rainforest armed with nothing but a spoon.

To cut – or scoop – through this jungle of music, Spotify’s research team deploy hundreds of machine learning models that improve the user experience, all the while trying to balance the needs of users and creators.

AI News caught up with Spotify research lead Rishabh Mehrotra at the AI & Big Data Expo Global on September 7 to learn more about how AI supports the platform.

AI News: How important is AI to Spotify’s mission?

Rishabh Mehrotra: AI is at the centre of what we do. Machine learning (ML) specifically has become an indispensable tool for powering personalised music and podcast recommendations to more than 365 million users across the world. It enables us to understand user needs and intents, which then helps us to deliver personalised recommendations across various touch points on the app.

It’s not just about the actual models which we deploy in front of users but also the various AI techniques we use to adopt a data driven process around experimentation, metrics, and product decisions.

We use a broad range of AI methods to understand our listeners, creators, and content. Some of our core ML research areas include understanding user needs and intents, matching content and listeners, balancing user and creator needs, using natural language understanding and multimedia information retrieval methods, and developing models that optimise long term rewards and recommendations.

What’s more, our models power experiences across around 180 countries, so we have to consider how they are performing across markets. Striking a balance between pushing global music but still facilitating local musicians and music culture is one of our most important AI initiatives.

AN: Spotify users might be surprised to learn just how central AI is to almost every aspect of the platform’s offering. It’s so seamless that I suspect most people don’t even realise it’s there. How crucial is AI to the user experience on Spotify?

RM: If you look at Spotify as a user then you typically view it as an app which gives you the content that you’re looking for. However, if you really zoom in you see that each of these different recommendation tools are all different machine learning products. So if you look at the homepage, we have to understand user intent in a far more subtle way than we would with a search query. The homepage is about giving recommendations based on a user’s current needs and context, which is very different from a search query where users are explicitly asking for something. Even in search, users can seek open and non-focused queries like ‘relaxing music’, or you could be searching the name of a specific song.

Looking at sequential radio sessions, our models try to balance familiar music with discovery content, aimed at not only recommending content users could enjoy at the moment, but optimising for long term listener-artist connections.

A good amount of our ML models are starting to become multi-objective. Over the past two years, we have deployed a lot of models that try to fulfil listener needs whilst also enabling creators to connect with and grow their audiences.

AN: Are artists’ wants and needs a big consideration for Spotify or is the focus primarily on the user experience?

RM: Our goal is to match the creators with the fans in an enriching way. While understanding user preferences is key to the success of our recommendation models, it really is a two-sided market in a lot of ways. We have the users who want to consume audio content on one side and the creators looking to grow their audiences on the other. Thus a lot of our recommendation products have a multi-stakeholder thinking baked into them to balance objectives from both sides.

AN: Apart from music recommendations and suggestions, does AI support Spotify in any other ways?

RM: AI plays an important role in driving our algotorial approach – Expert curators with an excellent sense for what’s up and coming, quite literally teach our machine learning system. Through this approach, we can create playlists that not only look at past data but also reflect cultural trends as they’re happening. Across all regions, we have editors who bring in deep domain expertise about music culture that we use proactively in our products. This allows us to develop and deploy human-in-the-loop AI techniques that can leverage editorial input to bootstrap various decisions that various ML models can then scale.

AN: What about podcasts? Do you utilise AI differently when applying it to podcasts over music?

RM: Users’ podcast journeys can differ in a lot of ways compared to music. While music is a lot about the audio and acoustic properties of songs, podcasts depend on a whole different set of parameters. For one, it’s much more about content understanding – understanding speakers, types of conversations and topics of discussions.

That said, we are seeing some very interesting results using music taste for podcast recommendations too. Members of our group have recently published work that shows how our ML models can leverage users’ music preferences to recommend podcasts, and some of these results have demonstrated significant improvements, especially for new podcast users.

AN: With so many models already turning the cogs at Spotify, it’s difficult to see how new and exciting use cases could be introduced. What are Spotify’s AI plans for the coming years?

RM: We’re working on a number of ways to elevate the experience even further. Reinforcement learning will be an important focus point as we look into ways to optimise for a lifetime of fulfilling content, rather than optimise for the next stream. In a sense this isn’t about giving users what they want right now as opposed to evolving their tastes and looking at their long term trajectories.

AN: As the years go on and your models have more and more data to work with, will the AI you use naturally become more advanced?

RM: A lot of our ML investments are not only about incorporating state-of-the-art ML into our products, but also extending the state-of-the-art based on the unique challenges we face as an audio platform. We are developing advanced causal inference techniques to understand the long term impact of our algorithmic decisions. We are innovating in the multi-objective ML modelling space to balance various objectives as part of our two-sided marketplace efforts. We are gravitating towards learning from long term trajectories and optimising for long term rewards.

To make data-driven decisions across all such initiatives, we rely heavily on solid scientific experimentation techniques, which also heavily relies on using machine learning.

Reinforcement learning furthers the scope of longer term decisions – it brings that long term perspective into our recommendations. So a quick example would be facilitating discovery on the platform. As a marketplace platform, we want users to not only consume familiar music but to also discover new music, leveraging the value of recommendations. There are 70 million tracks on the platform and only a few thousand will be familiar to any given user, putting aside the fact that it would take an individual several lifetimes to actually go through all this content. So tapping into that remaining 69.9 million and surfacing content users would love to discover is a key long-term goal for us.

How to fulfil users’ long term discovery needs, when to surface such discovery content, and by how much, not only across which set of users, but also across various recommended sets are a few examples of higher abstraction long term problems that RL approaches allow us to tackle well.

AN: Finally, considering the involvement Spotify has in directing users’ musical experiences, does the company have to factor in any ethical issues surrounding its usage of AI?

RM: Algorithmic responsibility and causal influence are topics we take very seriously and we actively work to ensure our systems operate in a fair and responsible manner, backed by focused research and internal education to prevent unintended biases.

We have a team dedicated to ensuring we approach these topics with the right research-informed rigour and we also share our learnings with the research community.

AN: Is there anything else you would like to share?

RM: On a closing note, one thing I love about Spotify is that we are very open with the wider industry and research community about the advances we are making with AI and machine learning. We actively publish at top tier venues, give tutorials, and we have released a number of large datasets to facilitate academic research on audio recommendations.

For anyone who is interested in learning more about this I would recommend checking out our Spotify Research website which discusses our papers, blogs, and datasets in greater detail.

Find out more about Digital Transformation Week North America, taking place on 9-10 November 2021, a virtual event and conference exploring advanced DTX strategies for a ‘digital everything’ world.

The post Rishabh Mehrotra, research lead, Spotify: Multi-stakeholder thinking with AI appeared first on AI News.

A Japanese artist will use a song partly composed by AI to welcome Pope Francis, who recently shared his concerns about the technology.

The song, “Protect all Life – The Signs of the Times,” is written by Jun Inoue.

Inoue is a Catholic himself and created an AI program which can generate a song in just a few seconds.

“I thought I should give everything I had to the song, so I decided to put in all the cutting-edge technology I had,” Inoue told Reuters.

Inoue wasn’t sure about using AI but decided to do so because of how intertwined the history of technology and music is. However, last month, AI News reported on Pope Francis’s concerns about artificial intelligence.

Pope Francis shared his views on AI during a Vatican conference attended by theologians, academics, and tech executives such as Mozilla co-founder Mitchell Baker and Facebook’s director of cybersecurity law Gavin Corn.

“If mankind’s so-called technological progress were to become an enemy of the common good, this would lead to an unfortunate regression to a form of barbarism dictated by the law of the strongest,” Pope Francis warned.

The Vatican hopes the conference will lead to a future document on considerations to make when developing AI technologies.

Inoue’s song is unlikely to trigger much concern from the Pope and may even help to show how AI can have a positive impact on people’s lives if used responsibly.

Earlier this year, a firm called Canny AI used deepfakes of politicians to create the following music video for John Lennon’s track Imagine:

This is, as the creators’ intended, an example of AI’s dangers when it comes to convincingly manipulating content.

Politicians and other influential figures could be made to appear like they’re saying and doing things which they’re not.

Such means could be used to sway public opinion and influence elections, and it’s going to be very difficult teaching the public not to necessarily believe their eyes and what to look out for when it comes to fake content.

Pope Francis will visit Japan from Nov 23rd-26th and marks only the second papal visit to the country.

Interested in hearing industry leaders discuss subjects like this? Attend the co-located 5G Expo, IoT Tech Expo, Blockchain Expo, AI & Big Data Expo, and Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London, and Amsterdam.

The post Japan will welcome Pope Francis with a song partly composed by AI appeared first on AI News.

Users can input their own melody and the AI will create a harmony in the Baroque style of Bach.

“Bach was a humble man who attributed his success to divine inspiration and a strict work ethic,” wrote Google in a post. “He lived to see only a handful of his works published, but more than 1,000 that survived in manuscript form are now published and performed all over the world.”

Aside from being a fun way of passing time, the doodle also intends to educate users on some basic fundamentals about how machine learning works.

Google’s model for its first AI-powered doodle was trained on 330 of Bach’s compositions. It was developed by Anna Huang from Google Magenta, in partnership with the Google PAIR (People + AI Research) team which provided TensorFlow expertise to allow the experience to run in just a browser.

Huang built Coconet, the model which powers this AI doodle that can harmonise melodies or compose them from scratch.

In a technical post explaining how Coconet works, the Magenta team wrote:

“Coconet is trained to restore Bach’s music from fragments: we take a piece from Bach, randomly erase some notes, and ask the model to guess the missing notes from context.

The result is a versatile model of counterpoint that accepts arbitrarily incomplete scores as input and works out complete scores.

This setup covers a wide range of musical tasks, such as harmonizing melodies, creating smooth transitions, rewriting and elaborating existing music, and composing from scratch.”

The doodle is available on Google’s homepage between Bach’s birthday (March 21st) to the 22nd.

Creating AIs is difficult, though arguably easier than putting 334 candles on a birthday cake to honour the man himself. Well (classically-)played, Google.

Interested in hearing industry leaders discuss subjects like this and their use cases? Attend the co-located AI & Big Data Expo events with upcoming shows in Silicon Valley, London, and Amsterdam to learn more. Co-located with the IoT Tech Expo, Blockchain Expo, and Cyber Security & Cloud Expo.

The post Become a modern-day Bach with Google’s AI-powered doodle appeared first on AI News.