Czyszczenie danych w Pythonie. Receptury. Nowoczesne techniki i narzędzia Pythona do wykrywania i eliminacji zanieczyszczeń oraz wydobywania kluczowych cech z danych

Michael Walker

Przetwarzanie dużych ilości danych daje wiedzę, która leży u podstaw istotnych decyzji podejmowanych przez organizację. Pozwala to na uzyskiwanie znakomitych efektów: techniki wydobywania wiedzy z danych stają się coraz bardziej wyrafinowane. Podstawowym warunkiem sukcesu jest uzyskanie odpowiedniej jakości danych. Wykorzystanie niespójnych i niepełnych informacji prowadzi do podejmowania błędnych decyzji. Konsekwencją mogą być straty finansowe, stwarzanie konkretnych zagrożeń czy uszczerbek na wizerunku. A zatem oczyszczanie jest wyjątkowo ważną częścią analizy danych. Ta książka jest praktycznym zbiorem gotowych do użycia receptur, podanych tak, aby maksymalnie ułatwić proces przygotowania danych do analizy. Omówiono tu takie kwestie dotyczące danych jak importowanie, ocena ich jakości, uzupełnianie braków, porządkowanie i agregacja, a także przekształcanie. Poza zwięzłym omówieniem tych zadań zaprezentowano najskuteczniejsze techniki ich wykonywania za pomocą różnych narzędzi: Pandas, NumPy, Matplotlib czy SciPy. W ramach każdej receptury wyjaśniono skutki podjętych działań. Cennym uzupełnieniem jest zestaw funkcji i klas zdefiniowanych przez użytkownika, które służą do automatyzacji oczyszczania danych. Umożliwiają one też dostrojenie procesu do konkretnych potrzeb. W książce znajdziesz receptury, dzięki którym: wczytasz i przeanalizujesz dane z różnych źródeł uporządkujesz dane, poprawisz ich błędy i uzupełnisz braki efektywnie skorzystasz z bibliotek Pythona zastosujesz wizualizacje do analizy danych napiszesz własne funkcje i klasy do automatyzacji procesu oczyszczania danych Prawdziwą wartość mają tylko oczyszczone i spójne dane!

Data Cleaning and Exploration with Machine Learning. Get to grips with machine learning techniques to achieve sparkling-clean data quickly

Michael Walker

Many individuals who know how to run machine learning algorithms do not have a good sense of the statistical assumptions they make and how to match the properties of the data to the algorithm for the best results.As you start with this book, models are carefully chosen to help you grasp the underlying data, including in-feature importance and correlation, and the distribution of features and targets. The first two parts of the book introduce you to techniques for preparing data for ML algorithms, without being bashful about using some ML techniques for data cleaning, including anomaly detection and feature selection. The book then helps you apply that knowledge to a wide variety of ML tasks. You’ll gain an understanding of popular supervised and unsupervised algorithms, how to prepare data for them, and how to evaluate them. Next, you’ll build models and understand the relationships in your data, as well as perform cleaning and exploration tasks with that data. You’ll make quick progress in studying the distribution of variables, identifying anomalies, and examining bivariate relationships, as you focus more on the accuracy of predictions in this book.By the end of this book, you’ll be able to deal with complex data problems using unsupervised ML algorithms like principal component analysis and k-means clustering.

Python Data Cleaning Cookbook. Modern techniques and Python tools to detect and remove dirty data and extract key insights

Michael Walker

Getting clean data to reveal insights is essential, as directly jumping into data analysis without proper data cleaning may lead to incorrect results. This book shows you tools and techniques that you can apply to clean and handle data with Python. You'll begin by getting familiar with the shape of data by using practices that can be deployed routinely with most data sources. Then, the book teaches you how to manipulate data to get it into a useful form. You'll also learn how to filter and summarize data to gain insights and better understand what makes sense and what does not, along with discovering how to operate on data to address the issues you've identified. Moving on, you'll perform key tasks, such as handling missing values, validating errors, removing duplicate data, monitoring high volumes of data, and handling outliers and invalid dates. Next, you'll cover recipes on using supervised learning and Naive Bayes analysis to identify unexpected values and classification errors, and generate visualizations for exploratory data analysis (EDA) to visualize unexpected values. Finally, you'll build functions and classes that you can reuse without modification when you have new data. By the end of this Python book, you'll be equipped with all the key skills that you need to clean data and diagnose problems within it.

Python Data Cleaning Cookbook. Prepare your data for analysis with pandas, NumPy, Matplotlib, scikit-learn, and OpenAI - Second Edition

Michael Walker

Jumping into data analysis without proper data cleaning will certainly lead to incorrect results. The Python Data Cleaning Cookbook - Second Edition will show you tools and techniques for cleaning and handling data with Python for better outcomes.Fully updated to the latest version of Python and all relevant tools, this book will teach you how to manipulate and clean data to get it into a useful form. he current edition focuses on advanced techniques like machine learning and AI-specific approaches and tools for data cleaning along with the conventional ones. The book also delves into tips and techniques to process and clean data for ML, AI, and NLP models. You will learn how to filter and summarize data to gain insights and better understand what makes sense and what does not, along with discovering how to operate on data to address the issues you've identified. Next, you’ll cover recipes for using supervised learning and Naive Bayes analysis to identify unexpected values and classification errors and generate visualizations for exploratory data analysis (EDA) to identify unexpected values. Finally, you’ll build functions and classes that you can reuse without modification when you have new data.By the end of this Data Cleaning book, you'll know how to clean data and diagnose problems within it.