Big data
Sinchan Banerjee
Java architectural patterns and tools help architects to build reliable, scalable, and secure data engineering solutions that collect, manipulate, and publish data.This book will help you make the most of the architecting data solutions available with clear and actionable advice from an expert.You’ll start with an overview of data architecture, exploring responsibilities of a Java data architect, and learning about various data formats, data storage, databases, and data application platforms as well as how to choose them. Next, you’ll understand how to architect a batch and real-time data processing pipeline. You’ll also get to grips with the various Java data processing patterns, before progressing to data security and governance. The later chapters will show you how to publish Data as a Service and how you can architect it. Finally, you’ll focus on how to evaluate and recommend an architecture by developing performance benchmarks, estimations, and various decision metrics.By the end of this book, you’ll be able to successfully orchestrate data architecture solutions using Java and related technologies as well as to evaluate and present the most suitable solution to your clients.
Tarik Makota, Brian Maguire, Danny Gagne, Rajeev...
Amazon Kinesis is a collection of secure, serverless, durable, and highly available purpose-built data streaming services. This data streaming service provides APIs and client SDKs that enable you to produce and consume data at scale.Scalable Data Streaming with Amazon Kinesis begins with a quick overview of the core concepts of data streams, along with the essentials of the AWS Kinesis landscape. You'll then explore the requirements of the use case shown through the book to help you get started and cover the key pain points encountered in the data stream life cycle. As you advance, you'll get to grips with the architectural components of Kinesis, understand how they are configured to build data pipelines, and delve into the applications that connect to them for consumption and processing. You'll also build a Kinesis data pipeline from scratch and learn how to implement and apply practical solutions. Moving on, you'll learn how to configure Kinesis on a cloud platform. Finally, you’ll learn how other AWS services can be integrated into Kinesis. These services include Redshift, Dynamo Database, AWS S3, Elastic Search, and third-party applications such as Splunk.By the end of this AWS book, you’ll be able to build and deploy your own Kinesis data pipelines with Kinesis Data Streams (KDS), Kinesis Data Firehose (KFH), Kinesis Video Streams (KVS), and Kinesis Data Analytics (KDA).
Scientific Computing with Python 3. Click here to enter text
Claus Führer, Jan Erik Solem, Olivier Verdier
Python can be used for more than just general-purpose programming. It is a free, open source language and environment that has tremendous potential for use within the domain of scientific computing. This book presents Python in tight connection with mathematical applications and demonstrates how to use various concepts in Python for computing purposes, including examples with the latest version of Python 3. Python is an effective tool to use when coupling scientific computing and mathematics and this book will teach you how to use it for linear algebra, arrays, plotting, iterating, functions, polynomials, and much more.
Julian Avila, Trent Hauck
Python is quickly becoming the go-to language for analysts and data scientists due to its simplicity and flexibility, and within the Python data space, scikit-learn is the unequivocal choice for machine learning. This book includes walk throughs and solutions to the common as well as the not-so-common problems in machine learning, and how scikit-learn can be leveraged to perform various machine learning tasks effectively.The second edition begins with taking you through recipes on evaluating the statistical properties of data and generates synthetic data for machine learning modelling. As you progress through the chapters, you will comes across recipes that will teach you to implement techniques like data pre-processing, linear regression, logistic regression, K-NN, Naïve Bayes, classification, decision trees, Ensembles and much more. Furthermore, you’ll learn to optimize your models with multi-class classification, cross validation, model evaluation and dive deeper in to implementing deep learning with scikit-learn. Along with covering the enhanced features on model section, API and new features like classifiers, regressors and estimators the book also contains recipes on evaluating and fine-tuning the performance of your model. By the end of this book, you will have explored plethora of features offered by scikit-learn for Python to solve any machine learning problem you come across.
John Sukup
Trusted by data scientists, ML engineers, and software developers alike, scikit-learn offers a versatile, user-friendly framework for implementing a wide range of ML algorithms, enabling the efficient development and deployment of predictive models in real-world applications. This third edition of scikit-learn Cookbook will help you master ML with real-world examples and scikit-learn 1.5 features.This updated edition takes you on a journey from understanding the fundamentals of ML and data preprocessing, through implementing advanced algorithms and techniques, to deploying and optimizing ML models in production. Along the way, you’ll explore practical, step-by-step recipes that cover everything from feature engineering and model selection to hyperparameter tuning and model evaluation, all using scikit-learn.By the end of this book, you’ll have gained the knowledge and skills needed to confidently build, evaluate, and deploy sophisticated ML models using scikit-learn, ready to tackle a wide range of data-driven challenges.*Email sign-up and proof of purchase required
Luiz Felipe Martins, V Kishore Ayyadevara, Ruben...
With the SciPy Stack, you get the power to effectively process, manipulate, and visualize your data using the popular Python language. Utilizing SciPy correctly can sometimes be a very tricky proposition. This book provides the right techniques so you can use SciPy to perform different data science tasks with ease.This book includes hands-on recipes for using the different components of the SciPy Stack such as NumPy, SciPy, matplotlib, and pandas, among others. You will use these libraries to solve real-world problems in linear algebra, numerical analysis, data visualization, and much more. The recipes included in the book will ensure you get a practical understanding not only of how a particular feature in SciPy Stack works, but also of its application to real-world problems. The independent nature of the recipes also ensure that you can pick up any one and learn about a particular feature of SciPy without reading through the other recipes, thus making the book a very handy and useful guide.
Vishal Pathak, Subramanya Vajiraya, Noritaka Sekiyama, Tomohiro...
Organizations these days have gravitated toward services such as AWS Glue that undertake undifferentiated heavy lifting and provide serverless Spark, enabling you to create and manage data lakes in a serverless fashion. This guide shows you how AWS Glue can be used to solve real-world problems along with helping you learn about data processing, data integration, and building data lakes.Beginning with AWS Glue basics, this book teaches you how to perform various aspects of data analysis such as ad hoc queries, data visualization, and real-time analysis using this service. It also provides a walk-through of CI/CD for AWS Glue and how to shift left on quality using automated regression tests. You’ll find out how data security aspects such as access control, encryption, auditing, and networking are implemented, as well as getting to grips with useful techniques such as picking the right file format, compression, partitioning, and bucketing. As you advance, you’ll discover AWS Glue features such as crawlers, Lake Formation, governed tables, lineage, DataBrew, Glue Studio, and custom connectors. The concluding chapters help you to understand various performance tuning, troubleshooting, and monitoring options.By the end of this AWS book, you’ll be able to create, manage, troubleshoot, and deploy ETL pipelines using AWS Glue.
Aaron Ploetz, Devram Kandhare, Sudarshan Kadambi, Xun...
This is the golden age of open source NoSQL databases. With enterprises having to work with large amounts of unstructured data and moving away from expensive monolithic architecture, the adoption of NoSQL databases is rapidly increasing. Being familiar with the popular NoSQL databases and knowing how to use them is a must for budding DBAs and developers.This book introduces you to the different types of NoSQL databases and gets you started with seven of the most popular NoSQL databases used by enterprises today. We start off with a brief overview of what NoSQL databases are, followed by an explanation of why and when to use them. The book then covers the seven most popular databases in each of these categories: MongoDB, Amazon DynamoDB, Redis, HBase, Cassandra, In?uxDB, and Neo4j. The book doesn't go into too much detail about each database but teachesyou enough to get started with them.By the end of this book, you will have a thorough understanding of the different NoSQL databases and their functionalities, empowering you to select and use the rightdatabase according to your needs.
Siatka danych. Nowoczesna koncepcja samoobsługowej infrastruktury danych
Zhamak Dehghani
Dostęp do danych jest warunkiem rozwoju niejednej organizacji. Aby w pełni skorzystać z ich potencjału i uzyskać dzięki nim konkretną wartość, konieczne jest odpowiednie zarządzanie danymi. Obecnie stosowane rozwiązania w tym zakresie nie nadążają już za złożonością dzisiejszych organizacji, rozprzestrzenianiem się źródeł danych i rosnącymi aspiracjami inżynierów, którzy rozwijają techniki sztucznej inteligencji i analizy danych. Odpowiedzią na te potrzeby może być siatka danych (Data Mesh), jednak praktyczna implementacja tej koncepcji wymaga istotnej zmiany myślenia. Ta książka szczegółowo wyjaśnia paradygmat siatki danych, a przy tym koncentruje się na jego praktycznym zastosowaniu. Zgodnie z tym nowatorskim podejściem dane należy traktować jako produkt, a dziedziny - jako główne zagadnienie. Poza wyjaśnieniem paradygmatu opisano tu zasady projektowania wysokopoziomowej architektury komponentów siatki danych, a także przedstawiono wskazówki i porady dotyczące ewolucyjnej realizacji siatki danych w organizacji. Tematyka ta została potraktowana wszechstronnie: omówiono kwestie technologiczne, organizacyjne, jak również socjologiczne i kulturowe. Dzięki temu jest to cenna lektura zarówno dla architektów i inżynierów, jak i dla badaczy, analityków danych, wreszcie dla liderów i kierowników zespołów. W książce: wyczerpujące wprowadzenie do paradygmatu siatki danych siatka danych i jej komponenty projektowanie architektury siatki danych opracowywanie i realizacja strategii siatki danych zdecentralizowany model własności danych przejście z hurtowni i jezior danych do rozproszonej siatki danych Siatka danych: kolejny etap rozwoju technologii big data!
Sakti Mishra
Amazon EMR, formerly Amazon Elastic MapReduce, provides a managed Hadoop cluster in Amazon Web Services (AWS) that you can use to implement batch or streaming data pipelines. By gaining expertise in Amazon EMR, you can design and implement data analytics pipelines with persistent or transient EMR clusters in AWS.This book is a practical guide to Amazon EMR for building data pipelines. You'll start by understanding the Amazon EMR architecture, cluster nodes, features, and deployment options, along with their pricing. Next, the book covers the various big data applications that EMR supports. You'll then focus on the advanced configuration of EMR applications, hardware, networking, security, troubleshooting, logging, and the different SDKs and APIs it provides. Later chapters will show you how to implement common Amazon EMR use cases, including batch ETL with Spark, real-time streaming with Spark Streaming, and handling UPSERT in S3 Data Lake with Apache Hudi. Finally, you'll orchestrate your EMR jobs and strategize on-premises Hadoop cluster migration to EMR. In addition to this, you'll explore best practices and cost optimization techniques while implementing your data analytics pipeline in EMR.By the end of this book, you'll be able to build and deploy Hadoop- or Spark-based apps on Amazon EMR and also migrate your existing on-premises Hadoop workloads to AWS.
Jomar Tigcal, Aileen Apolo-de Jesus
Coroutines and flows are the new recommended way for developers to carry out asynchronous programming in Android using simple, modern, and testable code. This book will teach you how coroutines and flows work and how to use them in building Android applications, along with helping you to develop modern Android applications with asynchronous programming using real data.The book begins by showing you how to create and handle Kotlin coroutines on Android. You’ll explore asynchronous programming in Kotlin, and understand how to test Kotlin coroutines. Next, you'll learn about Kotlin flows on Android, and have a closer look at using Kotlin flows by getting to grips with handling flow cancellations and exceptions and testing the flows.By the end of this book, you'll have the skills you need to build high-quality and maintainable Android applications using coroutines and flows.
Anindita Mahapatra, Doug May
Delta helps you generate reliable insights at scale and simplifies architecture around data pipelines, allowing you to focus primarily on refining the use cases being worked on. This is especially important when you consider that existing architecture is frequently reused for new use cases.In this book, you’ll learn about the principles of distributed computing, data modeling techniques, and big data design patterns and templates that help solve end-to-end data flow problems for common scenarios and are reusable across use cases and industry verticals. You’ll also learn how to recover from errors and the best practices around handling structured, semi-structured, and unstructured data using Delta. After that, you’ll get to grips with features such as ACID transactions on big data, disciplined schema evolution, time travel to help rewind a dataset to a different time or version, and unified batch and streaming capabilities that will help you build agile and robust data products.By the end of this Delta book, you’ll be able to use Delta as the foundational block for creating analytics-ready data that fuels all AI/BI use cases.
Simulation for Data Science with R. Effective Data-driven Decision Making
Matthias Templ
Data Science with R aims to teach you how to begin performing data science tasks by taking advantage of Rs powerful ecosystem of packages. R being the most widely used programming language when used with data science can be a powerful combination to solve complexities involved with varied data sets in the real world.The book will provide a computational and methodological framework for statistical simulation to the users. Through this book, you will get in grips with the software environment R. After getting to know the background of popular methods in the area of computational statistics, you will see some applications in R to better understand the methods as well as gaining experience of working with real-world data and real-world problems. This book helps uncover the large-scale patterns in complex systems where interdependencies and variation are critical. An effective simulation is driven by data generating processes that accurately reflect real physical populations. You will learn how to plan and structure a simulation project to aid in the decision-making process as well as the presentation of results.By the end of this book, you reader will get in touch with the software environment R. After getting background on popular methods in the area, you will see applications in R to better understand the methods as well as to gain experience when working on real-world data and real-world problems.
Skazany na sukces. Kariera w Data Science
Jacqueline Nolis, Emily Robinson
Nauka o danych, zwana danologią, zyskuje na znaczeniu. Dane dla gospodarki są tym, czym dotąd były węgiel, stal i ropa naftowa. Umiejętność korzystania z wiedzy zawartej w danych decyduje o efektywności prowadzenia działalności gospodarczej i determinuje rozwój nowych modeli, rozwiązań i relacji gospodarczych. Już teraz specjaliści danolodzy są rozchwytywani na rynku pracy. Aby jednak w pełni i do końca wykorzystać pojawiające się możliwości, trzeba wiedzieć, w jaki sposób podejść do trudnego zagadnienia, jakim jest budowanie ścieżki kariery i podążanie nią w odpowiednim dla siebie tempie. To praktyczny przewodnik, dzięki któremu łatwiej zdobędziesz pierwszą pracę związaną z badaniem danych, szybciej staniesz się cenionym specjalistą i w miarę rozwoju zawodowego będziesz coraz trafniej wychwytywać pojawiające się możliwości awansu i zmiany pracy na atrakcyjniejszą. Dowiesz się, jak zdobyć podstawowe umiejętności i jak faktycznie wyglądają konkretne stanowiska pracy. Opisano tu również, jak pomyślnie przejść przez proces rekrutacji i zaaklimatyzować się w nowych warunkach. Nie zabrakło cennych wskazówek dotyczących awansowania na stanowiska kierownicze. Jako danolog prędko się przekonasz, że zawarta tutaj wiedza nietechniczna jest bardzo potrzebna do osiągnięcia sukcesu na polu badania danych. Dzięki tej książce dowiesz się, jak: tworzyć świetne portfolio projektów z zakresu badania danych wyszukiwać, oceniać i negocjować oferty z klasą zmieniać miejsca pracy wybierać i skutecznie realizować scenariusze kariery poradzili sobie inni wybitni analitycy danych! Danologia: nauka, pasja i sposób na życie!
Slaying Excel Dragons. A Beginner's Guide to Conquering Excel's Frustrations and Making Excel Fun
MrExcel's Holy Macro! Books, Mike Girvin
This comprehensive guide is designed to elevate your Excel skills from beginner to advanced. Starting with the fundamentals, you'll learn how to navigate Excel's interface, use essential keyboard shortcuts, and manage data efficiently. As you progress, you'll dive into complex features like PivotTables, dynamic ranges, and advanced formatting, gaining the ability to handle intricate data tasks with ease.The guide also covers powerful formulas and functions, including VLOOKUP, INDEX/MATCH, and logical tests. These tools will empower you to automate calculations, perform detailed analyses, and streamline your workflow. Additionally, you'll explore Excel’s data analysis features, such as sorting, filtering, and creating dynamic charts, enabling you to present your data clearly and effectively.By the end of this book, you'll have a deep understanding of Excel's capabilities, equipped with the skills to tackle any spreadsheet challenge. Whether you're preparing for advanced data analysis or seeking to optimize your day-to-day tasks, this guide provides the knowledge and practical experience to make Excel work for you.