Making machine learning more useful to high-stakes decision makers

Asia's Tech News Daily

A visual analytics tool helps child welfare specialists understand machine learning predictions that can assist them in screening cases.

Research, Machine learning, Data, Human-computer interaction, Algorithms, Technology and society, Artificial intelligence, MIT Schwarzman College of Computing, School of Engineering, Electrical Engineering & Computer Science (eecs), Laboratory for Information and Decision Systems (LIDS)

A new visual analytics tool helps child welfare specialists understand machine learning predictions that can help them make decisions. Image: Christine Daniloff, MIT

The U.S. Centers for Disease Control and Prevention estimates that one in seven children in the United States experienced abuse or neglect in the past year. Child protective services agencies around the nation receive a high number of reports each year (about 4.4 million in 2019) of alleged neglect or abuse. With so many cases, some agencies are implementing machine learning models to help child welfare specialists screen cases and determine which to recommend for further investigation.

But these models don’t do any good if the humans they are intended to help don’t understand or trust their outputs.

Researchers at MIT and elsewhere launched a research project to identify and tackle machine learning usability challenges in child welfare screening. In collaboration with a child welfare department in Colorado, the researchers studied how call screeners assess cases, with and without the help of machine learning predictions. Based on feedback from the call screeners, they designed a visual analytics tool that uses bar graphs to show how specific factors of a case contribute to the predicted risk that a child will be removed from their home within two years.

The researchers found that screeners are more interested in seeing how each factor, like the child’s age, influences a prediction, rather than understanding the computational basis of how the model works. Their results also show that even a simple model can cause confusion if its features are not described with straightforward language.

These findings could be applied to other high-risk fields where humans use machine learning models to help them make decisions, but lack data science experience, says senior author Kalyan Veeramachaneni, principal research scientist in the Laboratory for Information and Decision Systems (LIDS) and senior author of the paper.

“Researchers who study explainable AI, they often try to dig deeper into the model itself to explain what the model did. But a big takeaway from this project is that these domain experts don’t necessarily want to learn what machine learning actually does. They are more interested in understanding why the model is making a different prediction than what their intuition is saying, or what factors it is using to make this prediction. They want information that helps them reconcile their agreements or disagreements with the model, or confirms their intuition,” he says.

Co-authors include electrical engineering and computer science PhD student Alexandra Zytek, who is the lead author; postdoc Dongyu Liu; and Rhema Vaithianathan, professor of economics and director of the Center for Social Data Analytics at the Auckland University of Technology and professor of social data analytics at the University of Queensland. The research will be presented later this month at the IEEE Visualization Conference.

Real-world research

The researchers began the study more than two years ago by identifying seven factors that make a machine learning model less usable, including lack of trust in where predictions come from and disagreements between user opinions and the model’s output.

With these factors in mind, Zytek and Liu flew to Colorado in the winter of 2019 to learn firsthand from call screeners in a child welfare department. This department is implementing a machine learning system developed by Vaithianathan that generates a risk score for each report, predicting the likelihood the child will be removed from their home. That risk score is based on more than 100 demographic and historic factors, such as the parents’ ages and past court involvements.

“As you can imagine, just getting a number between one and 20 and being told to integrate this into your workflow can be a bit challenging,” Zytek says.

They observed how teams of screeners process cases in about 10 minutes and spend most of that time discussing the risk factors associated with the case. That inspired the researchers to develop a case-specific details interface, which shows how each factor influenced the overall risk score using color-coded, horizontal bar graphs that indicate the magnitude of the contribution in a positive or negative direction.

Based on observations and detailed interviews, the researchers built four additional interfaces that provide explanations of the model, including one that compares a current case to past cases with similar risk scores. Then they ran a series of user studies.

The studies revealed that more than 90 percent of the screeners found the case-specific details interface to be useful, and it generally increased their trust in the model’s predictions. On the other hand, the screeners did not like the case comparison interface. While the researchers thought this interface would increase trust in the model, screeners were concerned it could lead to decisions based on past cases rather than the current report.   

“The most interesting result to me was that, the features we showed them — the information that the model uses — had to be really interpretable to start. The model uses more than 100 different features in order to make its prediction, and a lot of those were a bit confusing,” Zytek says.

Keeping the screeners in the loop throughout the iterative process helped the researchers make decisions about what elements to include in the machine learning explanation tool, called Sibyl.

As they refined the Sibyl interfaces, the researchers were careful to consider how providing explanations could contribute to some cognitive biases, and even undermine screeners’ trust in the model.

For instance, since explanations are based on averages in a database of child abuse and neglect cases, having three past abuse referrals may actually decrease the risk score of a child, since averages in this database may be far higher. A screener may see that explanation and decide not to trust the model, even though it is working correctly, Zytek explains. And because humans tend to put more emphasis on recent information, the order in which the factors are listed could also influence decisions.

Improving interpretability

Based on feedback from call screeners, the researchers are working to tweak the explanation model so the features that it uses are easier to explain.

Moving forward, they plan to enhance the interfaces they’ve created based on additional feedback and then run a quantitative user study to track the effects on decision making with real cases. Once those evaluations are complete, they can prepare to deploy Sibyl, Zytek says.

“It was especially valuable to be able to work so actively with these screeners. We got to really understand the problems they faced. While we saw some reservations on their part, what we saw more of was excitement about how useful these explanations were in certain cases. That was really rewarding,” she says.

This work is supported, in part, by the National Science Foundation.

Internet Explorer Channel Network
Asia's Tech News Daily
News Related

OTHER NEWS

Teenager Discovers 3,000-Year-Old Ax from the Bronze Age: 'We Were Just Laughing Our Heads Off'

Teenager Discovers 3,000-Year-Old Ax from the Bronze Age SWNS Milly Hardwick with the ax A day spent metal detecting took an exciting turn for one 13-year-old girl after she discovered ... Read more »

Accountant, 29, raped woman who thought she was having sex with his friend

Ibraheem Egunbambi had been turned down by the woman before he raped her (Reach) An accountant who raped a woman at a party while she thought she was having sex ... Read more »

The ‘world's most advanced,' realistic robot is here to terrify you

Get ready for Will Smith to say, “I told you so.” A scarily realistic robot with lifelike facial features and expressions that can “develop interaction” with people has just been ... Read more »

Arthur Labinjo-Hughes: ‘Cruel' stepmother who murdered boy is attacked with salt in jail by inmates

The stepmother and father of Arthur Labinjo-Hughes have been jailed over the six-year-old’s killing in Solihull (PA) An “instinctively cruel” stepmother convicted of murdering her partner's six-year-old son was threatened ... Read more »

Nicola Sturgeon blames Steps concert for omicron Covid outbreak

Steps singing on stage in 1999 – Nikos/Shutterstock Nicola Sturgeon has blamed a Steps concert for an outbreak of the omicron Covid variant in Scotland. The First Minister said the ... Read more »

Bereaved families ‘sickened' by No 10 lockdown parties

Boris Johnson in front of the Downing Street Christmas tree in 2020 (Justin Tallis/PA) Families who lost loved ones during the pandemic say they are “sickened” by reports that Downing ... Read more »

George Clooney turned down $35m for one day of work

Watch: George Clooney turned down $35m for one day of work George Clooney has revealed he once turned down $35 million (£29.8 million) for one day’s work. The actor has ... Read more »

Swiss district puts 2,000 people in quarantine after two Omicron cases at school

The entrance of the Châtaigneraie campus of the International School of Geneva which has closed after two cases of the Omicron variant were identified (Fabrice COFFRINI / AFP) More than ... Read more »

Minister claims Tory Xmas party backlash is 'rumour and hear'say' in excruciating exchange

Maggie Throup, the Minister for Vaccines and Public Health, was challenged about the Conservatives’ Christmas parties in 2020 but said the backlash surrounding the COVID rule-breaching events was ‘rumour and ... Read more »

New MP: Bexley by-election win 'greatest honour of my life'

The newest Conservative MP has promised to “get on delivering” for the people of Old Bexley and Sidcup, as he retained his party’s seat in the by-election following the death ... Read more »

Met not investigating No 10 over reported Christmas party, says Cressida Dick

Met not investigating No 10 over reported Christmas party, says Cressida Dick Read more »

Satellite image shows how vital ocean current is affected by climate crisis

A satellite image shows a major ocean current that is speeding up due to climate change. The Antarctic Circumpolar Current (ACC), which sweeps around the abyss off Antarctica, is the ... Read more »

The Webb Space Telescope Will Rewrite Cosmic History. If It Works.

The James Webb Space Telescope has been designed to answer many of the core questions that have animated astronomers over the past half-century. With a $10 billion price tag, it is one of the most ambitious engineering initiatives ever attempted. But for it to achieve its potential — nothing less than to rewrite the history of the cosmos and reshape humanity’s position within it — a lot of things have to work just right. Read more »

This Robot's Facial Expressions Will Freak You Out

Engineered Arts A U.K. robotics firm called Engineered Arts just debuted the first videos of its new humanoid robot, which is able to make hyper-realistic facial expressions. It’s a pretty ... Read more »

Save $150 on these noise-cancelling Sony headphones, along with more Best Buy Top Deals of the week

Looking for more Cyber Week deals, gift guides and holiday tips? Sign up for Yahoo Lifestyle Canada’s newsletter! These noise-cancelling Sony headphones are $150 off this week at Best Buy. ... Read more »

Diabetes drug may help improve symptoms for people with heart failure – new research

Half of people with heart failure have a type called preserved ejection fraction. Explode/ Shutterstock Heart failure is one of the most common cardiac diseases, affecting more than 30 million ... Read more »

Study suggests past COVID infection may not fend off omicron

An Orange Farm, South Africa, resident receives her jab against COVID-19 Friday Dec. 3, 2021 at the Orange Farm multipurpose center. South Africa has accelerated its vaccination campaign a week ... Read more »

Netflix Scoops Up ‘Three Songs for Benazir' for Doc Shorts Slate (EXCLUSIVE)

Netflix has bolstered its slate of Oscar short hopefuls with the acquisition of “Three Songs for Benazir.” Directed by Gulistan and Elizabeth Mirzaei, the 22-minute docu tells the story of ... Read more »

Omicron variant may increase risk of COVID reinfection, study finds

Johannesburg — As data on real-world infections in South Africa, where the Omicron variant of the coronavirus was first identified, suggest the mutated virus has an increased ability to reinfect ... Read more »

Government must back UK train travel or risk long-term retreat to cars

Jon Fitton/Shutterstock There’s no doubt that the UK’s railways have been hit hard by the pandemic. Recent figures from the Office of Rail and Road show a huge drop in ... Read more »

Robots doing brain surgery: CEO describes the latest in medical AI

The development of the Moderna (MRNA) coronavirus vaccine relied on it. The early discovery and modeling of COVID-19 outbreaks depended on it. And new drugs for the disease benefited from ... Read more »

Omicron covid variant three times more likely to cause reinfection than delta, South Africa study says

Omicron covid variant three times more likely to cause reinfection than delta, South Africa study says Scientists in South Africa say omicron is at least three times more likely to ... Read more »

Ghislaine Maxwell's 58-page rule book for staff at Jeffrey Epstein's mansion revealed

Ghislaine Maxwell Ghislaine Maxwell wrote a household manual for staff at Jeffrey Epstein’s Florida home, instructing employees to “see nothing, hear nothing, say nothing”, a former housekeeper revealed in testimony ... Read more »

Ghislaine Maxwell trial – live: Defence to quiz former Epstein employee after graphic sex toy testimony

Day five of the trial of British socialite Ghislaine Maxwell starts in New York City, as the 59-year-old answers to charges related to her alleged involvement in financier Jeffrey Epstein’s ... Read more »

Alcohol, anorexia and chronic colloidal silver ingestion revealed as cause of death in mummified Love Has Won cult leader

The autopsy report of Colorado cult leader Amy Carlson aka “Mother God” has revealed that the leader of religious group Love Has Won died due to a history of alcohol ... Read more »

Little Mix speak out on 'terribly emotional' hiatus decision

Watch: Little Mix 'taking break' after Confetti tour next year Little Mix have opened up on the “terribly emotional” decision to take a hiatus from the performing. This week saw ... Read more »

Trove of new dinosaur fossils in Italy helps rewrite prehistory of the Mediterranean region

An adult and two juvenile individuals of the dinosaur ‘Tethyshadros insularis’ showing the different appearances exhibited by immature and mature specimens (Davide Bonadonna) A trove of new dinosaur skeletons unearthed ... Read more »

Omicron: Britain plans to vaccinate 25 million in two months – but can it be done?

What impact the new omicron variant will have is still unknown. But with early data suggesting it could be the most transmissible form of the virus yet – and that ... Read more »

Having severe COVID-19 doubled the chance of dying within a year for people in a Florida study — the risk was even higher for young people

A COVID-19 patient recovering at home in Brooklyn, New York, on November 21, 2020.Andrew Lichtenstein/Corbis/Getty Images A new study tracked nearly 180 hospitalized COVID-19 patients in Florida for up to ... Read more »

California has 50 times more monarch butterflies than last year — a puzzling blip in their path to extinction

Butterflies land on branches at Monarch Grove Sanctuary in Pacific Grove, California, on November 10, 2021.Nic Coury/AP Photo Monarch butterflies migrating to California made a promising rebound from the edge ... Read more »

Scientists Have Found The Two Most Effective Covid Booster Jabs

(Photo: Yui Mok – PA Images via Getty Images) The Pfizer and Moderna vaccines are the most effective Covid booster jabs. a new study has found, but all boosters could ... Read more »

Indian doctor with no travel history tests positive for omicron: ‘This may be everywhere already'

Indian doctor with no travel history tests positive for omicron: ‘This may be everywhere already’ A 46-year-old Indian doctor with no travel history is among the two people who have ... Read more »

Omicron: better to be safe (and quick) than sorry

People Image Studio/Shutterstock On discovering the omicron variant, many countries moved quickly to impose travel restrictions and other public health measures, such as compulsory mask wearing. But, given the lack ... Read more »

Psst: This Amazon Fire HD 10 tablet bundle is an absurd $90 right now

Yahoo Life has received compensation to create this article, and receives commission from purchases made via links on this page. Pricing and availability are subject to change. Take your pick ... Read more »

Brits won't serve Christmas finger food because of Covid, study finds

© Photo by Sasha • Stories on Unsplash canapes-134408-unsplash.jpg If you thought it was hors-d’over for finger foods – it is now. Brits will shun canapés at Christmas this year ... Read more »

Sky Glass review: does the broadband provider's new TV launch live up to the hype?

(Sky) Not content with being just an internet or TV provider, Sky has ventured into devices with its latest launch. Europe’s leading media and entertainment company has unveiled the Sky ... Read more »

Gene edited sex selection may spare animal suffering

Mice have been genetically altered to produce either male- or female-only offspring. Scientists have used gene editing technology to create female-only and male-only litters of mice. Their technique could prevent ... Read more »

Black Friday may be over, but these Apple AirPod deals are still running

Apple AirPods are still the must-have headphones (Getty) Yahoo Entertainment's editors are committed to independently selecting wonderful products at great prices for you. We may receive a share from purchases ... Read more »

Pigs beaten and cut without pain relief on farm linked to Tesco, Sainsbury, Morrisons and Asda

Pigs were beaten, mutilated and kept in “squalid” conditions on a French farm linked to Tesco, Sainsbury’s, Morrisons, Asda and Iceland, investigators say. Undercover footage suggested some animals were suffering ... Read more »

Elon Musk now controls more than a third of all active satellites as SpaceX breaks rocket launch record

A live satellite orbit visualisation captured on 1 December, 2021. A string of SpaceX Starlink satellites can be seen crossing the Indian Ocean off the coast of Madagascar (CelesTrak) The ... Read more »
On free-english-test.com you will find lots of free English exam practice materials to help you improve your English skills: grammar, listening, reading, writing, ielts, toeic