From sketch to symphony: How AI and a team of experts finished Beethoven's 10th

© Crice Denyer/CC BY

Beethoven was commissioned in 1817 to write two symphonies: he completed his Ninth Symphony in 1824, but the 10th remained in its early stages at the time of his death three years later

– Crice Denyer/CC BY

Ludwig van Beethoven died in 1827, three years after completing his Ninth Symphony, which many consider his masterpiece. He had started to work on a 10th Symphony, but the deterioration of his state of health did not allow him to complete this project, which remained in the draft stage.

Since then, musicologists and Beethoven lovers have wondered – and lament not knowing – what this symphony could have become. Now, thanks to the work of a team of music historians, musicologists, composers and computer engineers, Beethoven’s vision has come to life.

Responsible for the artificial intelligence (AI) side of the project, I led a scientific team from the AI start-up Playform that taught a machine Beethoven’s entire body of work and his creative process.

A full recording of his 10th Symphony was released on 9 October 2021, and was performed on stage for the first time in Bonn, Germany, crowning more than two years of effort.

Past attempts hit a wall

© Provided by The Independent To create something that Beethoven could have written, the AI ​​had to know all of the composer’s work (Hulton Fine Art Collection / Getty Images)

Around 1817, the Royal Philharmonic Society in London commissioned Beethoven to write his Ninth and 10th Symphonies. Written for an orchestra, symphonies have four movements: the first is played at a fast tempo, the second at a slower tempo, the third at a moderate or fast tempo, and the last at a fast tempo.

In 1824, Beethoven completed his Ninth Symphony, which concludes with the timeless “Ode to Joy”.

When it came to the 10th Symphony, however, the composer left little behind, other than some musical notes and a handful of ideas he had jotted down.

There have been some past attempts to reconstruct parts of Beethoven’s 10th Symphony. Most famously, in 1988, the musicologist Barry Cooper ventured to complete the first and second movements. He wove together 250 bars of music from the sketches to create what was, in his view, a production of the first movement that was faithful to Beethoven’s vision.

Yet the sparseness of Beethoven’s sketches made it impossible for symphony experts to go beyond that first movement.

Assembling the team

In early 2019, I was contacted by Dr Matthias Roeder, the director of the Karajan Institute, an organisation in Salzburg, Austria, that aims to promote the links between music and technology.

He explained that he was putting together a team to complete Beethoven’s 10th Symphony in celebration of the 250th anniversary of the composer’s birth. Aware of my work on AI-generated art, he wanted to know if AI could help fill in the blanks left by Beethoven.

The challenge seemed daunting. To pull it off, AI would need to do something it had never done before. But I replied that I was ready to give it a try.

Roeder then formed a team that included the Austrian composer Walter Werzowa. Famous for writing Intel’s signature jingle, Werzowa was tasked with putting together a new kind of composition that would integrate what Beethoven left behind with what the AI would generate.

© Provided by The Independent A page of Beethoven’s notes for his planned 10th Symphony (Beethoven House Museum , CC BY-SA)

Mark Gotham, a computational music expert, led the effort to transcribe Beethoven’s sketches and process his entire body of work so the AI could be properly trained.

The team also included Robert Levin, a musicologist at Harvard University who also happens to be an incredible pianist. Levin had previously finished a number of incomplete 18th-century works by Mozart and Johann Sebastian Bach.

The project takes shape

In June 2019, the group gathered for a two-day workshop at Harvard’s music library. In a large room with a piano, a blackboard and a stack of Beethoven’s sketchbooks spanning most of his known works, we talked about how fragments could be turned into a complete piece of music and how AI could help solve this puzzle, while still remaining faithful to Beethoven’s process and vision.

The music experts in the room were eager to learn more about the sort of music AI had created in the past. I told them how AI had successfully generated music in the style of Bach. However, this was only a harmonisation of an inputted melody that sounded like Bach. It didn’t come close to what we needed to do: construct an entire symphony from a handful of phrases.

Meanwhile, the scientists in the room – myself included – wanted to learn about what sort of materials were available, and how the experts envisioned using them to complete the symphony.

At every point, Beethoven’s genius loomed, challenging us to do better. As the project evolved, the AI did as well

The task at hand eventually crystallised. We would need to use notes and completed compositions from Beethoven’s entire body of work – along with the available sketches from the 10th Symphony – to create something that Beethoven himself might have written.

This was a tremendous challenge. We didn’t have a machine that we could feed sketches to, push a button and have it spit out a symphony. Most AI available at the time couldn’t continue an uncompleted piece of music beyond a few additional seconds.

We would need to push the boundaries of what creative AI could do by teaching the machine Beethoven’s creative process – how he would take a few bars of music and painstakingly develop them into stirring symphonies, quartets and sonatas.

Piecing together Beethoven’s creative process

As the project progressed, the human side and the machine side of the collaboration evolved. Werzowa, Gotham, Levin, and Roeder deciphered and transcribed the sketches from the 10th Symphony, trying to understand Beethoven’s intentions. Using his completed symphonies as a template, they attempted to piece together the puzzle of where the fragments of sketches should go – which movement, which part of the movement.

They had to make decisions, such as determining whether a sketch indicated the starting point of a scherzo, which is a very lively part of the symphony, typically in the third movement. Or they might determine that a line of music was likely the basis of a fugue, which is a melody created by interweaving parts that all echo a central theme.

The AI side of the project – my side – found itself grappling with a range of challenging tasks.

First, and most fundamentally, we needed to figure out how to take a short phrase, or even just a motif, and use it to develop a longer, more complicated musical structure, just as Beethoven would have done. For example, the machine had to learn how Beethoven constructed the Fifth Symphony out of a basic four-note motif.

One of the music experts said the AI reminded him of an eager music student who practices every day, learns, and becomes better and better

Next, because the continuation of a phrase also needs to follow a certain musical form, whether it’s a scherzo, trio or fugue, the AI needed to learn Beethoven’s process for developing these forms.

The to-do list grew: we had to teach the AI how to take a melodic line and harmonise it. The AI needed to learn how to bridge two sections of music together. And we realised the AI had to be able to compose a coda, which is a segment that brings a section of a piece of music to its conclusion.

Finally, once we had a full composition, the AI was going to have to figure out how to orchestrate it, which involves assigning different instruments for different parts.

And it had to pull off these tasks in the way Beethoven might have done.

Passing the first big test

In November 2019, the team met in person again – this time at the Beethoven House Museum, in Bonn, where the composer was born and raised.

This meeting was the litmus test for determining whether AI could complete this project. We printed musical scores that had been developed by AI and built off the sketches from Beethoven’s 10th. A pianist performed in a small concert hall in the museum before a group of journalists, music scholars and Beethoven experts.

We challenged the audience to determine where Beethoven’s phrases ended and where the AI extrapolation began. They couldn’t.

© Provided by The Independent Journalists and musicians gather to hear a pianist play excerpts from the piece (Ahmed Elgammal/CC BY-SA)

A few days later, one of these AI-generated scores was played by a string quartet at a news conference. Only those who intimately knew Beethoven’s sketches for the 10th Symphony could determine when the AI-generated parts came in.

The success of these tests told us we were on the right track. But these were just a couple of minutes of music. There was still much more work to do.

Ready for the world

At every point, Beethoven’s genius loomed, challenging us to do better. As the project evolved, the AI did as well. Over the ensuing 18 months, we constructed and orchestrated two entire movements of more than 20 minutes apiece.

We anticipate some pushback to this work – those who will say that the arts should be off-limits from AI, and that AI has no business trying to replicate the human creative process. Yet when it comes to the arts, I see AI not as a replacement, but as a tool – one that opens doors for artists to express themselves in new ways.

This project would not have been possible without the expertise of human historians and musicians. It took an immense amount of work – and, yes, creative thinking – to accomplish this goal.

At one point, one of the music experts on the team said that the AI reminded him of an eager music student who practices every day, learns, and becomes better and better.

Now that student, having taken the baton from Beethoven, has presented the 10th Symphony to the world.

Ahmed Elgammal is a professor and the director of the Art & AI Lab at Rutgers University. This article first appeared on The Conversation.

From news to politics, travel to sport, culture to climate – The Independent has a host of free newsletters to suit your interests. To find the stories you want to read, and more, in your inbox, click here.

Internet Explorer Channel Network
News Related

OTHER NEWS

LNG demand to rise 25-50% by 2030: Morgan Stanley

Samsung Heavy Industries’ LNG carrier / Courtesy of Samsung Heavy Industries Demand for liquefied natural gas (LNG) is expected to rise by 25 percent to 50 percent by 2030, making ... Read more »

Lenovo launches Tab K10 with 10.3-inch FHD display, MediaTek Helio P22T and 7500 mAh battery at Rs 25,000

Lenovo has announced the launch of its new tablet Tab K10 In India. The new Android 11-based tablet comes with a 10.3-inch FHD display. The Lenovo Tab K10 is powered ... Read more »

Wu-Tang Clan could lead official soundtrack's production for rumored Xbox game codenamed ‘Shaolin'

Photo credit: Billy Freeman / Unsplash It has been more than a month since a massive list of unannounced games were leaked from the Nvidia GeForce Now network, but fans continue hearing that ... Read more »

One of Microsoft's most-hated products might actually be getting a useful upgrade

Searching for lost files or that extra bit of data to make the difference at work could be quicker than ever thanks to a new upgrade from Microsoft. The tech ... Read more »

Best wireless gaming mouse 2021

There was a time when even the best wireless gaming mouse couldn’t hold a candle to its tethered colleagues, but that’s all in the past. The wireless gaming mice of ... Read more »

Twitter downvote feature may come to web as well

Twitter is reportedly working to expand the downvote feature to more users. According to the known tech blogger, Jane Manchun Wong, the company is working on a downvote button for ... Read more »

macOS Monterey rolling out on October 25: Here are the things you should keep in mind before updating

Announced at Worldwide Developers Conference, macOS Monterey will be rolling out to compatible Mac laptops and desktops on October 25. The operating system brings several new features and changes to ... Read more »

Google Drive for desktop client receives complete support for Apple M1 chip

Google has finally updated its Drive for desktop application with full support for Apple M1 powered Mac laptops and desktops.Earlier this year, Google rolled out M1 support for its previous ... Read more »

Best web development tools in 2021

The best web development tools make it simple and easy for developers to develop and design a custom website platform. Web development has long since left the grounds of having ... Read more »

Best Excel online courses in 2021

The best Excel online courses will make it easy to develop basic or advanced skills using Microsoft Excel by distance learning. Best Excel online courses Click the links below to ... Read more »

Facebook papers: What we know about how misinformation and extremism spread from whistleblower

Misinformation and extremism spreading unchecked. Hate speech sparking conflict and violence in the U.S. and abroad. Human traffickers sharing a platform with baby pictures and engagement announcements. Despite its mission ... Read more »

The Facebook Papers: Documents reveal internal fury and dissent over site's policies

Thousands of leaked documents highlight employees’ disillusionment with spread of misinformation and calls to violence. Read more »

Huawei CFO Meng Wanzhou returns to hero's welcome at company headquarters after 21-day quarantine

Huawei Technologies Co finance chief Meng Wanzhou, who has completed a 21-day Covid-19 quarantine after returning from Canada, came back to her company’s head office to a hero’s welcome on ... Read more »

Electric vehicle savings explored

Driving an electric vehicle can save you a lot of money on gas… but how much money does it really save you? Well, it depends. Read more »

Why Strictly Come Dancing fans are fuming with Anton Du Beke

Photo credit: BBC Strictly Come Dancing fans had two major talking points after Saturday evening's [23rd October] show: Adam Peaty's fairly tight trousers, and Anton Du Beke's mispronunciation of Oti ... Read more »

Oklahoma resumes lethal injections that ‘burn men alive' this week. Seven men may die before they can appeal

Oklahoma may execute seven death row inmates before their legal appeal in federal court can play out. (Images courtesy of Oklahoma Department of Corrections and Justice for Julius; Composite by ... Read more »

Museum bosses back anti-woke charter to guard against rushed and unplanned changes to UK heritage

A statue of Robert Milligan is pictured being removed by workers outside the Museum of London Docklands near Canary Wharf – Reuters Bosses from three of the UK’s leading museums ... Read more »

Megan Thee Stallion's bodyguard snubs F1 interviewer who awkwardly asked her to ‘freestyle rap'

Megan Thee Stallion’s bodyguard snubbed F1 interviewer Martin Brundle after he asked the singer to “freestyle rap”. Footage of the awkward encounter was broadcast on Sky Sports, as the rapper ... Read more »

Facebook whistleblower live: Frances Haugen appears before UK parliamentary committee

The latest on Facebook as data scientist turned whistleblower Frances Haugen prepares to give evidence to MPs on the UK parliament's Joint Committee on the Draft Online Safety Bill from 2.30pm. Read more »

Another popular npm package infected with malware

In an audacious incident, threat actors hijacked the account of the developer of a widely used JavaScript library, UAParser.ja, to replace the legitimate code with malicious one infused with malware ... Read more »

Best Raspberry Pi distros in 2021

Believe it or not, the Raspberry Pi is now eight years old. In this relatively short period the Pi has ushered in a new revolution in computing that stretches far ... Read more »

Elon Musk reveals his crypto buys

Elon Musk said he holds Dogecoin but no Shiba Inu. Photo: AP Tesla (TSLA) CEO Elon Musk, a crypto proponent, revealed he has not invested in meme token Shiba Inu, ... Read more »

'World's most expensive' fish spotted off coast of Britain

‘World’s most expensive’ fish spotted off coast of Britain Read more »

Panasonic unveils prototype battery to help Tesla lower production costs

Panasonic on Monday unveiled a new large prototype battery designed to help Tesla Inc lower electric vehicle (EV) production costs, in a move the Japanese company’s battery chief said would ... Read more »

A world without checkouts: How till-free supermarkets are changing London's grocery game

(ES composite) Last time I walked out of a shop without paying, I had to make a red-faced trip back to Boots. This time — at a Tesco Express by ... Read more »

Electric cars: Price of EVs puts people off switching from petrol and diesel, poll finds

The cost of electric vehicles is the biggest factor putting people off switching from petrol and diesel, exclusive polling for i has revealed. According to the survey, conducted by Redfield ... Read more »

Julian Knight: Online Safety Bill is ‘simplistic and bad politics', Tory MP tells Boris Johnson

Banning anonymous social media accounts is “simplistic and unworkable”, the chair of the Commons Culture, Media and Sport committee has told i. Julian Knight also took aim at Boris Johnson ... Read more »

Nintendo Switch doesn't need better hardware, according to World War Z dev

With the polarizing release of the Nintendo Switch OLED leading many Switch owners to stick with their original models, interest in an eventual upgraded version remains high. A potential Nintendo ... Read more »

Tesla Cybertruck looks unbelievably cool in new off-road footage

Earlier this year, Tesla CEO Elon Musk described his company’s in-development Cybertruck as a “special project” near and dear to his heart – and thanks to some brand new footage ... Read more »

Idea Spectrum Realtime Landscaping Pro 2020 review

Landscape design is usually back breaking work, involving a spade, and most likely a wheelbarrow. But why not forgo all of this and start your work from the comfort of ... Read more »

5 things I wished I knew when I started playing New World

One of the first lessons you learn is to not call this fella “Bob”. (Screenshot: Amazon Games Studio) Aside from some technical issues, Amazon Games Studio's New World, its massively ... Read more »

Five ways to drive business impact from big data

Big data has been a favorite business buzzwords for many years and while the popularity of the term skyrocketed, some were already tired of it by 2014, like Deloitte who ... Read more »

Ethereum's upgrade is finally coming — will it overtake Bitcoin?

The world’s second most valuable cryptocurrency, ether, has been touching all-time highs in price ahead of a major upgrade of its underlying platform, ethereum. Ether is currently worth in aggregate ... Read more »

Best mobile payment apps in 2021

The best mobile payment apps offer a simple way to make and send payments using your smartphone. Mobile apps providing contactless payments have become common since the introduction of Near ... Read more »

‘eFootball 2022' update: Konami delays version 0.9.1 release date to November

Photo credit: Konami via eFootball channel / YouTube screenshot This year marked a massive shift in Konami’s football simulation franchise, but the developer has yet to regain the trust of ... Read more »

Shark Anti Hair Wrap Upright Vacuum Cleaner with Powered Lift-Away NZ850UK review

One-minute review Shark has become a well-known brand in the floorcare market, and offers an array of models from uprights to cylinders, corded, cordless, and even handheld options – no ... Read more »

JioPhone Next key specs and features confirmed: All you need to know

Reliance Jio has released a video detailing its vision and the joint effort with Google and Qualcomm it invested in the making of the JioPhone Next. The short film also confirmed the features and key ... Read more »

Fortnite skins October 2021: All the skins coming to Fortnite and how to get them

October 2021 is here and that means Fortnite is getting spooky with a whole host of new Halloween-inspired skins for players to wear in-game. We know it’s hard to keep ... Read more »

Intel Alder Lake leak shows a laptop CPU which outperforms Apple M1 Max

Intel’s Alder Lake processors for laptops are expected to arrive in Q1 of 2022, following the launch of 12th-gen desktop chips which is purportedly imminent, but when these mobile CPUs ... Read more »

Scientists find one of the youngest planets ever

(University of Hawaii) Scientists have found one of the youngest planets ever discovered, having only been formed a few million years ago. The planet, 2M0437b, is several times larger than ... Read more »
On free-english-test.com you will find lots of free English exam practice materials to help you improve your English skills: grammar, listening, reading, writing, ielts, toeic