How quickly do algorithms improve?

MIT scientists show how fast algorithms are improving across a broad range of examples, demonstrating their critical importance in advancing computing.

Algorithms, Technology and society, Computer science and technology, Computing, Data, Software, Electrical Engineering & Computer Science (eecs), Computer Science and Artificial Intelligence Laboratory (CSAIL), School of Engineering, Sloan School of Management, MIT Schwarzman College of Computing, the new paper

MIT scientists present the first systematic, quantitative evidence that algorithms are one of the most important sources of improvement in computing. Image: Degui Adil/EyeEm

Algorithms are sort of like a parent to a computer. They tell the computer how to make sense of information so they can, in turn, make something useful out of it.

The more efficient the algorithm, the less work the computer has to do. For all of the technological progress in computing hardware, and the much debated lifespan of Moore’s Law, computer performance is only one side of the picture.

Behind the scenes a second trend is happening: Algorithms are being improved, so in turn less computing power is needed. While algorithmic efficiency may have less of a spotlight, you’d definitely notice if your trusty search engine suddenly became one-tenth as fast, or if moving through big datasets felt like wading through sludge.

This led scientists from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) to ask: How quickly do algorithms improve?  

Existing data on this question were largely anecdotal, consisting of case studies of particular algorithms that were assumed to be representative of the broader scope. Faced with this dearth of evidence, the team set off to crunch data from 57 textbooks and more than 1,110 research papers, to trace the history of when algorithms got better. Some of the research papers directly reported how good new algorithms were, and others needed to be reconstructed by the authors using “pseudocode,” shorthand versions of the algorithm that describe the basic details.

In total, the team looked at 113 “algorithm families,” sets of algorithms solving the same problem that had been highlighted as most important by computer science textbooks. For each of the 113, the team reconstructed its history, tracking each time a new algorithm was proposed for the problem and making special note of those that were more efficient. Ranging in performance and separated by decades, starting from the 1940s to now, the team found an average of eight algorithms per family, of which a couple improved its efficiency. To share this assembled database of knowledge, the team also created Algorithm-Wiki.org.

The scientists charted how quickly these families had improved, focusing on the most-analyzed feature of the algorithms — how fast they could guarantee to solve the problem (in computer speak: “worst-case time complexity”). What emerged was enormous variability, but also important insights on how transformative algorithmic improvement has been for computer science.

For large computing problems, 43 percent of algorithm families had year-on-year improvements that were equal to or larger than the much-touted gains from Moore’s Law. In 14 percent of problems, the improvement to performance from algorithms vastly outpaced those that have come from improved hardware. The gains from algorithm improvement were particularly large for big-data problems, so the importance of those advancements has grown in recent decades.

The single biggest change that the authors observed came when an algorithm family transitioned from exponential to polynomial complexity. The amount of effort it takes to solve an exponential problem is like a person trying to guess a combination on a lock. If you only have a single 10-digit dial, the task is easy. With four dials like a bicycle lock, it’s hard enough that no one steals your bike, but still conceivable that you could try every combination. With 50, it’s almost impossible — it would take too many steps. Problems that have exponential complexity are like that for computers: As they get bigger they quickly outpace the ability of the computer to handle them. Finding a polynomial algorithm often solves that, making it possible to tackle problems in a way that no amount of hardware improvement can.

As rumblings of Moore’s Law coming to an end rapidly permeate global conversations, the researchers say that computing users will increasingly need to turn to areas like algorithms for performance improvements. The team says the findings confirm that historically, the gains from algorithms have been enormous, so the potential is there. But if gains come from algorithms instead of hardware, they’ll look different. Hardware improvement from Moore’s Law happens smoothly over time, and for algorithms the gains come in steps that are usually large but infrequent. 

“This is the first paper to show how fast algorithms are improving across a broad range of examples,” says Neil Thompson, an MIT research scientist at CSAIL and the Sloan School of Management and senior author on the new paper. “Through our analysis, we were able to say how many more tasks could be done using the same amount of computing power after an algorithm improved. As problems increase to billions or trillions of data points, algorithmic improvement becomes substantially more important than hardware improvement. In an era where the environmental footprint of computing is increasingly worrisome, this is a way to improve businesses and other organizations without the downside.”

Thompson wrote the paper alongside MIT visiting student Yash Sherry. The paper is published in the Proceedings of the IEEE. The work was funded by the Tides foundation and the MIT Initiative on the Digital Economy.

Internet Explorer Channel Network
News Related

OTHER NEWS

Female African elephants evolved toward being tuskless over just a few decades as poachers sought ivory

A tusked African elephant in South Africa’s Kruger National Park. Getty During the civil war in Mozambique, armies hunted African elephants to near extinction to collect ivory tusks. A study ... Read more »

Map shows areas of the UK with the highest CO2 emissions

The City of London produced the most CO2 in the country. (Getty) The City of London produces more than twice as much carbon dioxide (CO2) than anywhere else in the ... Read more »

The UK's 20 COVID hotspots as cases surge above 50,000 per day

The government has resisted calls to implement plan B. (PA) The UK has recorded more than 50,000 cases of COVID-19 for the first time in months as large areas of ... Read more »

Suzanne Somers poses pantsless to ring in 75th birthday: 'Turning three quarters of a century with love in my life feels great!'

Suzanne Somers is celebrating another year around the sun — this time, without pants. The Three’s Company alum took to Instagram on Wednesday to share a photo of herself in ... Read more »

Mohamed Noor: Officer who shot dead 911 caller reporting rape is jailed for five years

A former Minneapolis police officer who shot a 911 caller dead has been sentenced to almost five years in prison. Mohamed Noor was convicted of manslaughter in the death of ... Read more »

Bid to unionize Amazon workers in New York nears milestone

NEW YORK (AP) — A bid to unionize Amazon workers at a distribution facility in New York City neared an important milestone, as organizers prepared to deliver hundreds of signatures ... Read more »

Researchers have found why some people are fans of ultra-rich individuals like Jeff Bezos or Elon Musk, but not billionaires in general

A mobile billboard calling for higher taxes on the ultra-wealthy with an image of billionaire Jeff Bezos, near the US Capitol on May 17. Drew Angerer/Getty Images People tend to ... Read more »

Ivory poaching has led to evolution of tuskless elephants, study finds

Photograph: Idaho Statesman/MCT via Getty Images Ivory poaching over decades has led to the evolution of tuskless elephants, researchers have found, proving that humans are “literally changing the anatomy” of ... Read more »

Facebook's oversight board seeks details on VIPs' treatment

WASHINGTON (AP) — Facebook’s semi-independent oversight board says the company has failed to fully disclose information on its internal system that exempts high-profile users from some or all of its ... Read more »

Asteroids May Have Stolen The Oxygen From Earth's Ancient Atmosphere

For a period of Earth’s history, between roughly 2.5 to 4 billion years ago, our planet was a punching bag for asteroids. © SwRI/Dan Durda, Simone Marchi Artist’s impression of ... Read more »

Judi Love to miss Strictly Come Dancing this weekend as she tests Covid-19 positive

(PA) Strictly Come Dancing hopeful Judi Love has tested positive for Covid-19 and she will miss this weekend’s show. The Loose Women star, who lives in London with her two ... Read more »

Manchester Arena bombing inquiry: Attacker's brother 'takes coward's way out' after leaving UK, victims' families say

The brother of the Manchester Arena bomber has “taken the coward's way out”, families of his victims have said, as lawyers accepted he had been allowed to “laugh in the ... Read more »

Whether it's cock-up or conspiracy, officials are still feeding the public misleading statistics

Sajid Javid speaks next to Stephen Powis, NHS England National Medical Director and UK Health Security Agency Chief Executive, Dr Jenny Harries during a press conference – Toby Melville/WPA Pool ... Read more »

How the ‘Dune' science-fiction saga parallels the real science of Oregon's dunes

Timothy Chalamet and Rebecca Ferguson star in “Dune,” a movie based on sci-fi author Frank Herbert’s book. The tale draws in part upon Herbert’s experiences in the Pacific Northwest. (Warner ... Read more »

Bite-Sized Brains Grown in a Lab Could Help Treat Fatal Diseases

Photo Illustration by Elizabeth Brockway/The Daily Beast/Getty It has all the makings of a Frankenstein-esque horror movie about science gone wrong: A group of scientists huddled over small organs vaguely ... Read more »

Regulator to probe into tech giants' payment practices

FILE – In this May 8, 2019, file photo, Federal Trade Commission commissioner Rohit Chopra testifies during a House Energy and Commerce subcommittee hearing on Capitol Hill in Washington. The ... Read more »

Floods in India's north leave dozens dead

Officials conducted an aerial survey over the Himalayan state of Uttarakhand, which showed the extent of the destruction. Destroyed food items, clothes, electrical appliances and vehicles were seen drying in ... Read more »

William Shatner's Family Tried Talking Him Out of Space Flight — But He Said 'Save Your Breath'

william shatner Cover Images/Zuma As Captain Kirk on Star Trek, William Shatner became synonymous with space exploration in the 1960s — but several decades later, his family wasn't thrilled when ... Read more »

The Brain Processes Speech in Parallel With Other Sounds

Scientists thought that the brain’s hearing centers might just process speech along with other sounds. But new work suggests that speech gets some special treatment very early on. Read more »

There's an ideal window to get your flu shot for the best protection: late October or the first week of November

A man walks past an ad for free flu shots outside a CVS drugstore in New York on August 19, 2020. Bryan R. Smith/AFP/Getty Images The CDC recommends getting a ... Read more »

Police hunting two ‘aggressive' women who smashed wine bottle over Tube passenger's head

Police are searching for two women after a Tube passenger was hit over the head with a wine bottle. (SWNS) British Transport Police (BTP) are appealing for help in finding ... Read more »

Hundreds Sickened After Onion-Linked Salmonella Outbreak

(Bloomberg) — People across the U.S. have been advised to throw away all unlabeled red, white and yellow onions after a mass salmonella outbreak sickened hundreds of people across 37 ... Read more »

Spectacular Flaming Space Debris Confuses Skywatchers Amid Orionid Meteor Shower

A “bright, long-lasting” fireball that sailed over Highland, Michigan, in the early morning of October 20, coinciding with the peak of the Orionid meteor shower, was in fact a fragmented ... Read more »

Dinosaurs may have lived in social herds as early as 193 million years ago

Fossils indicate a communal nesting ground and adults who foraged and took care of the young as a herd, scientists say. Read more »

Lab-grown 'mini brains' are developed from the cells of dementia and motor neurone patients – opening the door to new treatments for the neurodegenerative conditions

Researchers made the mini brains and had them survive for up to 340 daysThese models of a brain made from real cells act as a testing lab for new drugsThe team say future mini brains can be made and customised to individualsThis will allow for medication to be tested against real world symptoms  Read more »

What Europe's exceptionally low winds mean for the future energy grid

Shaggyphoto / shutterstock Through summer and early autumn 2021, Europe experienced a long period of dry conditions and low wind speeds. The beautifully bright and still weather may have been ... Read more »

How 'mix and match' COVID-19 booster shots work, and why we're using them

Chanei Henry, senior research coordinator of molecular virology and microbiology at Baylor College of Medicine, prepares a COVID-19 booster shot during a mix-and-match trial. Baylor College of Medicine Fully vaccinated ... Read more »

Only two DOZEN parts of England have yet to record new Delta off-shoot

© Provided by Daily Mail MailOnline logo Fewer than two dozen areas of England are yet to record a single case of the new Delta off-shoot, official figures suggest. One ... Read more »

Pfizer booster dose offers exceptionally high levels of protection against Covid and all variants

(Getty Images) A booster dose of the Pfizer/BioNTech vaccine offers exceptionally high levels of protection against coronavirus, including the Delta variant, new trial data show. Pfizer said its trial of ... Read more »

Deep-space radiation could cause have big impacts on the brain, mouse experiment shows

Could deep-space radiation seriously affect the human brain?  © Provided by Space This visualization shows space radiation interacting with Earth’s magnetosphere. In a new study, male mice exposed to simulated ... Read more »

A Capitol riot suspect who said he was 'invincible' because he was wearing a mask has now been arrested

Capitol riot suspect Landon Mitchell (circled) FBI Landon Mitchell boasted that he would likely evade detection because he was “masked up,” a complaint said. He told friends he was “invincible,” ... Read more »

Launch of new London to Edinburgh train service

Launch of new London to Edinburgh train service Read more »

Jack Nicholson makes first public appearance in two years

Jack Nicholson attended a basketball game with and his son Ray last night. (Kevork Djansezian/Getty Images) Jack Nicholson made a rare public appearance last night, attending a Lakers game with ... Read more »

Boris Johnson and Jacinda Arden announce post-Brexit trade deal

Boris Johnson and Jacinda Arden announce post-Brexit trade deal Read more »

Cause of mysterious deaths of California family to be revealed two months after bodies were found along trail

(KSEE/KGPE by Rosanna Heaslett) The mystery surrounding the deaths of a California family will finally be put to rest on Thursday, two months after their bodies were found along a ... Read more »

Jacob Rees Mogg: Tory MPs don't need masks because of our ‘convivial spirit'

Credit: Parliament Live TV Read more »

Flooding in South of England as Low-Pressure System Moves In

Flooding hit various parts of southern England on the night of October 20-21 as a low-pressure system moved in from the continent brining heavy rain and strong winds. Footage here ... Read more »
On free-english-test.com you will find lots of free English exam practice materials to help you improve your English skills: grammar, listening, reading, writing, ielts, toeic