Six ways to improve data science in the cloud

Asia's Tech News Daily

The data science market is flourishing, with an increasing number of companies across sectors placing data at the forefront of their digital transformation strategies. The rise of data analytics has seen the demand for data scientists and data engineers tripling over the past five years, rising by as much as 231%. Yet as many businesses rush to hire the talent they need to make their plans a reality, many are still on a journey in realizing the full value that their data can offer.

Six ways to improve data science in the cloud
© Provided by TechRadar Holographic representation of cloud computing over open businessman’s hand

Organizations that previously used legacy architecture often face challenges when attempting to retrofit their systems to the cloud. As a result, it can be difficult to adapt, and habits and biases from the on-premises world can limit the understanding of what’s possible in the cloud. Data scientists, data engineers and developers are all having to acclimate to their new cloud environments and a rapidly evolving ecosystem of tools and frameworks. With many learning on the job, businesses risk not maximizing the potential of their cloud architecture.

If harnessed correctly, the cloud can revolutionize data science and create an exciting frontier for companies to better understand customers, monetize data in new ways, and make predictions about the future. Data teams now have access to a vast pool of elastic computing power, as well as numerous sources of internal and external data. Managed cloud services are also available to reduce the complexity of building, training and deploying machine learning and deep learning models at scale. Here are six tangible strategies that businesses can follow to make the most of data science in the cloud.

1. Don’t compromise on data governance

It’s critical for businesses to enable iteration and investigation into data without compromising governance and security. Before they start working on a dataset, many data scientists intuitively want to make a copy of the original. But all too often copies are made and forgotten about, creating problems when it comes to compliance, security and privacy. A modern data platform should enable data teams to work on snapshots, or virtual copies, without needing to duplicate entire datasets, all while maintaining fine-grained controls to ensure that only the right users and applications have access to the data in hand. Businesses must create processes that minimize duplications to ensure internal and external data governance policies are being met.

2. Start with what you want to achieve

Pre-existing biases from operating on-premises can often hold businesses back and hinder them from focusing on what they wish to achieve with their data. For example, one common misconception is when data scientists say: “I’d love to retrain my model several times a day, but it’s too slow and will delay other processes.” But that’s not an issue in a world of elastic infrastructure. When migrating to the cloud, it’s therefore important for companies to recognize the full breadth of new capabilities on offer in order to dispel any previous biases that are no longer relevant within a cloud model.

Removing any preconceived perceptions will empower businesses, to make the most of their data and be ambitious. Once in this position, data teams must start with what they want to achieve, not what they think is possible, and move forward from there. That’s the only way they can push boundaries and take full advantage of the cloud.

3. Create a single source of truth

Closely tied to data governance is the concept of silos – these occur when data sits separately from each other, meaning no one person or team in an organization has a holistic view of all the data in its possession. The proliferation of tools, platforms and vendors is great for innovation, but it can also lead to redundant, inconsistent data being stored in multiple locations. Another cause of fragmentation is when structured data is stored in one environment, such as a data warehouse, while semi-structured data ends up in a data lake. Besides compromising governance and security, this fragmentation can get in the way of achieving better predictions or classifications.

To tackle data silos head on, organizations should work with a cloud data platform that provides a global, consolidated view of their data. That means a platform that can accommodate structured, semi-structured and unstructured data side by side. It also means a platform that can provide a single instance of this data across multiple cloud providers and tools — not six versions of data replicated across different platforms and environments.

4. Exploit new tools and technologies

One of the exciting things about data science is that frameworks and tools are evolving at an incredible pace, but it’s critical for businesses to not get locked into an approach that limits their options when technologies fall in and out of favour. To give one example: Spark ML used to be the answer to most large-scale training problems, but now TensorFlow and PyTorch are capturing the most attention. Businesses never know what will happen next year, or next week for that matter. They should choose a data platform that won’t tie them into one framework or one way of doing things, one that has an architecture that can grow as they do and accommodate new tools and technologies as they come along.

5. Embrace third-party data

The cloud makes it much easier to incorporate external data from partners and data-service providers into models. This was particularly important over the past year, as businesses sought to understand how the impact of COVID-19, fluctuations in the economy, and subsequent changes in consumer behavior, would affect their businesses. For example, organizations used data about local infection rates, foot traffic in stores and insights from social media to predict consumer buying patterns and forecast inventory needs. By doing so, they were able to adjust product inventory and staff numbers to meet customer demand.

Capital One is just one example of a business successfully harnessing Snowflake’s Data Marketplace to access and share data with third parties quickly and securely. The bank uses Starschema, a third-party data provider, as part of its broader efforts to understand the impact of COVID-19. This helped the company to forecast and plan response scenarios for its workforce and its customers. This is something all businesses should take note of – by exploring the numerous external data sources available to them, organizations can better answer some of their most pressing business needs.

6. Don’t overcomplicate the process

It’s often said that when you have a hammer, everything looks like a nail, and this applies to AI technologies like machine learning and deep learning. They are immensely powerful and have a critical role to play for certain business needs, but they’re not the answer to every problem. Businesses should always start with the simplest option and increase complexity as needed. Try a simple linear regression, or look at averages and medians. How accurate are the predictions? Does the ROI of increasing the accuracy justify a more complex approach? Sometimes it does, but don’t jump to that option as your first instinct. Starting with a simple approach will help data teams transition from legacy architecture with ease and make the most of the ensuing benefits, such as third party data.

Once companies leave behind their preconceptions about the cloud and exploit its full potential, they will create an environment that delivers enhanced and dynamic data analytics. This will empower data teams to better understand their customers and create new revenue streams by monetizing their data. As the data science market continues to grow, now is the time to address the challenges of migrating to the cloud and improve the way analytics is deployed and used.

Internet Explorer Channel Network
Asia's Tech News Daily
News Related


PlayStation plans new game subscription service to take on Xbox Game Pass

Currently, PlayStation Plus is required for most online multiplayer games and offers free monthly titles. NEW YORK (BLOOMBERG) – Sony Group’s PlayStation division is planning a new subscription service to ... Read more »

What AI researchers can learn from the NFL Combine

How useful is your AI? It’s not a simple question to answer. If you’re trying to decide between Google‘s Translate service or Microsoft’s Translator, how do you know which one ... Read more »

Use the Verizon app? Check your settings to protect your privacy

If you are a Verizon customer and use the wireless carrier’s mobile app, you might want to give your privacy settings a close check. Verizon’s app includes access to Customer ... Read more »

A third of online Black Friday shoppers were completely fake

If you had trouble scoring that new smart TV or Xbox Series X this Black Friday, bots and fake users could be to blame according to a new report from ... Read more »

NASA's 10 new astronauts: pilots, doctor, physicist, cyclist

The Associated PressThis photo provided by NASA shows its 2021 astronaut candidate class, announced on Monday, Dec. 6, 2021. The 10 candidates stand for a photo at the Johnson Space ... Read more »

Deal that would bring Donald Trump's media company to the stock market under investigation

NEW YORK — Regulators are looking into the deal that would bring Donald Trump’s new social media company to the stock market, one that has attracted both legions of fans ... Read more »

The 50 best Christmas gifts for everyone on your list

— Recommendations are independently chosen by Reviewed’s editors. Purchases you make through our links may earn us a commission. The Christmas season is finally here and if you’re looking to ... Read more »

Blockchain Underdogs Set Sight On Changing Internet Infrastructure

The internet is an essential commodity that is evolving and reshaping faster than anybody’s estimation. From desktop to desktop communication of the Web 1.0 era –when it was all about ... Read more »

Cybersecurity is a priority for Latin American consumers: report

A study commissioned by Mastercard to Kantar revealed that cybersecurity has become more important to consumers as they enter the digital economy more. According to the “Digital Security Barometer” revealed ... Read more »

Creating a Visual Language for Innovative Products

We take many things in our environment for granted: materials, textures, shadows and reflections. Sometimes we think of these visual attributes in isolation, but the true power is when they ... Read more »

10 of the best job search sites to use during your job hunt

— Recommendations are independently chosen by Reviewed’s editors. Purchases you make through our links may earn us a commission. Hunting for a new gig? Thankfully, your perfect workplace may be ... Read more »

The best headphones of 2021 are on sale for a record-low price of Amazon

If you’re looking to score a discount on some of the best headphones on the market, then you’re in luck. Amazon has the Sony WH-1000XM4 noise cancelling headphones on sale ... Read more »

Updated daily: Here are the 5 best Amazon deals today in December 2021

— Recommendations are independently chosen by Reviewed’s editors. Purchases you make through our links may earn us a commission. Every day, we scour Amazon to find the best deals available. ... Read more »

Careful where you click: Tragedies like Michigan's Oxford school shooting attract scammers

Tragedy can unite communities, shine a light on heroes and remind us once again that life is indeed incredibly fragile. And tragedy opens the door for the scammers who know ... Read more »

Patchy monitoring means UK Omicron numbers unclear, say officials

Photograph: Andy Rain/EPA Only 30-35% of PCR tests for general population are processed at labs that look for a key indicator of new variant The Omicron variant of coronavirus is ... Read more »

Russia to send Japanese space tourist tycoon to ISS

Yusaku Maezawa has been training for his space mission for months (ASSOCIATED PRESS) A Japanese billionaire will blast into space aboard a Russian rocket on Wednesday, marking Moscow’s return to ... Read more »

Chinese lander spots cube-shaped 'mystery hut' on the dark side of the moon

The Chinese Yutu-2 lunar rover just spotted a cube-shaped “mystery hut” along the horizon as it explores the far side of the moon and is currently driving over to investigate. ... Read more »

Google is launching a more secure place for your photos

Keeping the photos and videos stored on your Android smartphone away from prying eyes is about to get a whole lot easier as the new Locked Folder feature in Google ... Read more »

Bus drivers' brainwaves will be monitored in South Korean safety pilot

This article was originally published by  on , the leading news platform on urban mobility and innovation, reaching an international audience of city leaders. For the latest updates, follow Cities Today on , , , , ... Read more »

30 gifts for women under $100 to invest in for the 2021 holiday season

— Recommendations are independently chosen by Reviewed’s editors. Purchases you make through our links may earn us a commission. The holidays are quickly approaching, and many of us are looking ... Read more »

A Stargazer's Guide to the 2021 Geminid Meteor Shower: When It Will Peak and How to Watch

gremind meteor Getty Mark your calendars and set your alarms because the Geminid meteor shower is upon us, expecting to peak during the late evening of Dec. 13 into the ... Read more »

Best SD Cards 2021: the best memory cards for your video or stills camera

Searching for the best SD cards for your camera? You’ve come to the right place. We’ve speed-tested, reviewed and ranked all of the best choices around for stills and video ... Read more »

Dark matter is definitely missing in mysterious galaxy, study says

(Javier Román & Pavel Mancera Piña) A mysterious galaxy is definitely missing its dark matter, the authors of a new paper say. The scientists behind the research say that when ... Read more »

Human intuition and machine reasoning: AI's helping us solve giant math puzzles

Research in mathematics is a deeply imaginative and intuitive process. This might come as a surprise for those who are still recovering from high-school algebra. What does the world look ... Read more »

Tesla Under Investigation By SEC Over SolarCity Panels Defects

Tesla Inc (NASDAQ:TSLA) is facing scrutiny by the Securities and Exchange Commission (SEC) over fire risks caused by alleged SolarCity’s solar panel defects. A whistleblower complained that the firm had ... Read more »

How to install any screen protector on any camera | Sony, Canon, Nikon

Welcome back to The Ultimate Tech Hub. On today’s episode we will install a LCD Screen protector for our camera. We will install the Glass Protector from Expert Shield onto ... Read more »

Crypto exchange BitMart vows to compensate for $205 million hack

NEW YORK (BLOOMBERG) – Crypto exchange BitMart will use its own funding to compensate users affected in a security breach that saw hackers withdraw about US$150 million (S$205 million) of ... Read more »

Google Pixel phones are getting another round of feature updates

Back when Google launched the Pixel 6 and the Pixel 6 Pro, we heard about a few features that were “coming soon” – and some of those features are now ... Read more »

Activision Blizzard's Call Of Duty game testers plan walkout over job cuts

NEW YORK (BLOOMBERG) – A group of workers at an Activision Blizzard division supporting the Call Of Duty franchise plan to call out of work Monday (Dec 6) in protest ... Read more »

Forza Motorsport car list, news and rumors

Forza Motorsport is the upcoming eighth entry in Microsoft’s long-running racing game franchise and while we don’t know when it’ll release just yet we’re expecting it to be a true ... Read more »

Starfield: release date, trailers, exclusivity and everything you need to know

Starfield, the long-awaited new IP from Bethesda Game Studios, is coming to Xbox PC in 2022 and it looks set to take us on an epic sci-fi adventure. Starfield details ... Read more »

The Xbox Windows app will soon tell you how well games will play on your PC

Microsoft is testing a new feature in the Xbox App for Windows that will let you know how well a game should run on your PC. The feature is currently ... Read more »

Snag a Gravity Blanket for a massive discount during this huge holiday sale

— Recommendations are independently chosen by Reviewed’s editors. Purchases you make through our links may earn us a commission. If you’re searching for a calming gift that will offer your ... Read more »

WhatsApp is adding this new functionality to disappearing messages feature

WhatsApp is giving users more control for its disappearing messages feature. Facebook-owned WhatsApp says that users will now have the option to turn on disappearing messages automatically for all their ... Read more »

TikTok secret algorithm is a big 'Duh!'

TikTok’s powerful algorithm steering the social media activity of millions of daily users is no longer a mystery. In a new New York Times report that outlines information gleaned from ... Read more »

BioWare confirms commitment to making Dragon Age 4 ‘single-player focused'

Dragon Age 4 developer, BioWare, has confirmed that the game will be a “single-player focused experience.” The confirmation comes by way of a recent blog post celebrating Dragon Age Day, ... Read more »

Lack of Workforce Insight Is Killing Your Business

There’s no denying that the future of work is all about flexibility. As employees, we love being able to replace lengthy commutes with more focused time when working from home, ... Read more »

You can now be part of the new social network, Woonkly, which will be based on NFt and metaverses

The new social network Woonkly seeks to bring together the world of Blockchain and reality. Unlike the social networks that we already have, Woonkly is based on NFT and metaverses ... Read more »

How AI Solutions Can Improve the Travel and Hotel Booking Experience

Regardless of what Covid-19 has in store this winter, the travel industry is looking ahead to 2022 and remaining hopeful for a robust recovery. But even as travel bounces back, ... Read more »

Shop Ulta Beauty's Holiday Beauty Blitz and take 50% off makeup, hair and skincare gifts

— Recommendations are independently chosen by Reviewed’s editors. Purchases you make through our links may earn us a commission. Holiday shopping just got a festive makeover thanks to the month-long ... Read more »
On you will find lots of free English exam practice materials to help you improve your English skills: grammar, listening, reading, writing, ielts, toeic