Azure Data Engineering Cookbook. Design and implement batch and streaming analytics using Azure Cloud Services

Ahmad Osama

Data engineering is one of the faster growing job areas as Data Engineers are the ones who ensure that the data is extracted, provisioned and the data is of the highest quality for data analysis. This book uses various Azure services to implement and maintain infrastructure to extract data from multiple sources, and then transform and load it for data analysis.It takes you through different techniques for performing big data engineering using Microsoft Azure Data services. It begins by showing you how Azure Blob storage can be used for storing large amounts of unstructured data and how to use it for orchestrating a data workflow. You'll then work with different Cosmos DB APIs and Azure SQL Database. Moving on, you'll discover how to provision an Azure Synapse database and find out how to ingest and analyze data in Azure Synapse. As you advance, you'll cover the design and implementation of batch processing solutions using Azure Data Factory, and understand how to manage, maintain, and secure Azure Data Factory pipelines. You’ll also design and implement batch processing solutions using Azure Databricks and then manage and secure Azure Databricks clusters and jobs. In the concluding chapters, you'll learn how to process streaming data using Azure Stream Analytics and Data Explorer.By the end of this Azure book, you'll have gained the knowledge you need to be able to orchestrate batch and real-time ETL workflows in Microsoft Azure.

Azure Data Scientist Associate Certification Guide. A hands-on guide to machine learning in Azure and passing the Microsoft Certified DP-100 exam

Andreas Botsikas, Michael Hlobil

The Azure Data Scientist Associate Certification Guide helps you acquire practical knowledge for machine learning experimentation on Azure. It covers everything you need to pass the DP-100 exam and become a certified Azure Data Scientist Associate.Starting with an introduction to data science, you'll learn the terminology that will be used throughout the book and then move on to the Azure Machine Learning (Azure ML) workspace. You'll discover the studio interface and manage various components, such as data stores and compute clusters.Next, the book focuses on no-code and low-code experimentation, and shows you how to use the Automated ML wizard to locate and deploy optimal models for your dataset. You'll also learn how to run end-to-end data science experiments using the designer provided in Azure ML Studio.You'll then explore the Azure ML Software Development Kit (SDK) for Python and advance to creating experiments and publishing models using code. The book also guides you in optimizing your model's hyperparameters using Hyperdrive before demonstrating how to use responsible AI tools to interpret and debug your models. Once you have a trained model, you'll learn to operationalize it for batch or real-time inferences and monitor it in production.By the end of this Azure certification study guide, you'll have gained the knowledge and the practical skills required to pass the DP-100 exam.

Bayesian Analysis with Python. A practical guide to probabilistic modeling - Third Edition

Osvaldo Martin, Christopher Fonnesbeck, Thomas Wiecki

The third edition of Bayesian Analysis with Python serves as an introduction to the main concepts of applied Bayesian modeling using PyMC, a state-of-the-art probabilistic programming library, and other libraries that support and facilitate modeling like ArviZ, for exploratory analysis of Bayesian models; Bambi, for flexible and easy hierarchical linear modeling; PreliZ, for prior elicitation; PyMC-BART, for flexible non-parametric regression; and Kulprit, for variable selection.In this updated edition, a brief and conceptual introduction to probability theory enhances your learning journey by introducing new topics like Bayesian additive regression trees (BART), featuring updated examples. Refined explanations, informed by feedback and experience from previous editions, underscore the book's emphasis on Bayesian statistics. You will explore various models, including hierarchical models, generalized linear models for regression and classification, mixture models, Gaussian processes, and BART, using synthetic and real datasets.By the end of this book, you will possess a functional understanding of probabilistic modeling, enabling you to design and implement Bayesian models for your data science challenges. You'll be well-prepared to delve into more advanced material or specialized statistical modeling if the need arises.

Bayesian Analysis with Python. Introduction to statistical modeling and probabilistic programming using PyMC3 and ArviZ - Second Edition

Osvaldo Martin

The second edition of Bayesian Analysis with Python is an introduction to the main concepts of applied Bayesian inference and its practical implementation in Python using PyMC3, a state-of-the-art probabilistic programming library, and ArviZ, a new library for exploratory analysis of Bayesian models.The main concepts of Bayesian statistics are covered using a practical and computational approach. Synthetic and real data sets are used to introduce several types of models, such as generalized linear models for regression and classification, mixture models, hierarchical models, and Gaussian processes, among others. By the end of the book, you will have a working knowledge of probabilistic modeling and you will be able to design and implement Bayesian models for your own data science problems. After reading the book you will be better prepared to delve into more advanced material or specialized statistical modeling if you need to.

Become a Python Data Analyst. Perform exploratory data analysis and gain insight into scientific computing using Python

Alvaro Fuentes

Python is one of the most common and popular languages preferred by leading data analysts and statisticians for working with massive datasets and complex data visualizations.Become a Python Data Analyst introduces Python’s most essential tools and libraries necessary to work with the data analysis process, right from preparing data to performing simple statistical analyses and creating meaningful data visualizations.In this book, we will cover Python libraries such as NumPy, pandas, matplotlib, seaborn, SciPy, and scikit-learn, and apply them in practical data analysis and statistics examples. As you make your way through the chapters, you will learn to efficiently use the Jupyter Notebook to operate and manipulate data using NumPy and the pandas library. In the concluding chapters, you will gain experience in building simple predictive models and carrying out statistical computation and analysis using rich Python tools and proven data analysis techniques.By the end of this book, you will have hands-on experience performing data analysis with Python.

Becoming an Enterprise Django Developer. Discover best practices, tooling, and solutions for writing and organizing Django applications in production

Michael Dinder

Django is a powerful framework but choosing the right add-ons that match the scale and scope of your enterprise projects can be tricky. This book will help you explore the multifarious options available for enterprise Django development. Countless organizations are already using Django and more migrating to it, unleashing the power of Python with many different packages and dependencies, including AI technologies.This practical guide will help you understand practices, blueprints, and design decisions to put Django to work the way you want it to. You’ll learn various ways in which data can be rendered onto a page and discover the power of Django for large-scale production applications. Starting with the basics of getting an enterprise project up and running, you'll get to grips with maintaining the project throughout its lifecycle while learning what the Django application lifecycle is.By the end of this book, you'll have learned how to build and deploy a Django project to the web and implement various components into the site.

Before Machine Learning Volume 2 - Calculus for A.I. The Fundamental Mathematics for Data Science and Artificial Intelligence

Jorge Brasil

This book takes readers on a structured journey through calculus fundamentals essential for AI. Starting with “Why Calculus?” it introduces key concepts like functions, limits, and derivatives, providing a solid foundation for understanding machine learning.As readers progress, they will encounter practical applications such as Taylor Series for curve fitting, gradient descent for optimization, and L'Hôpital’s Rule for managing undefined expressions. Each chapter builds up from core calculus to multidimensional topics, making complex ideas accessible and applicable to AI.The final chapters guide readers through multivariable calculus, including advanced concepts like the gradient, Hessian, and backpropagation, crucial for neural networks. From optimizing models to understanding cost functions, this book equips readers with the calculus skills needed to confidently tackle AI challenges, offering insights that make complex calculus both manageable and deeply relevant to machine learning.

Beginning Data Science with Python and Jupyter. Use powerful industry-standard tools within Jupyter and the Python ecosystem to unlock new, actionable insights from your data

Alex Galea

Get to grips with the skills you need for entry-level data science in this hands-on Python and Jupyter course. You'll learn about some of the most commonly used libraries that are part of the Anaconda distribution, and then explore machine learning models with real datasets to give you the skills and exposure you need for the real world. We'll finish up by showing you how easy it can be to scrape and gather your own data from the open web, so that you can apply your new skills in an actionable context.

Bezpieczeństwo sieci w Pythonie. Rozwiązywanie problemów za pomocą skryptów i bibliotek. Wydanie II

José Manuel Ortega

Popularność Pythona wynika z jego wszechstronności, prostoty, a także ze zwięzłości i z łatwości pisania kodu. Rozbudowywana z każdą aktualizacją kolekcja narzędzi i bibliotek pozwala na używanie Pythona do coraz bardziej specjalistycznych zadań, takich jak zabezpieczanie sieci. O tym, że skuteczna ochrona sieci ma krytyczne znaczenie dla organizacji, świadczą powtarzające się przypadki cyberataków i utraty cennych danych. Warto więc wykorzystać możliwości Pythona do wykrywania zagrożeń i rozwiązywania różnych problemów związanych z siecią. Tę książkę docenią specjaliści do spraw bezpieczeństwa i inżynierowie sieci. Dzięki niej zapoznasz się z najnowszymi pakietami i bibliotekami Pythona i nauczysz się pisać skrypty, które pozwolą Ci zabezpieczyć sieć na wielu poziomach. Dowiesz się, w jaki sposób przesyłać dane i korzystać z sieci Tor. Nauczysz się też identyfikować podatności systemu na ataki, aby tym skuteczniej zapewnić mu bezpieczeństwo. W naturalny sposób przyswoisz wiedzę, która pozwoli Ci tworzyć w Pythonie bezpieczne aplikacje, zaczniesz również stosować techniki kryptograficzne i steganograficzne. Znajdziesz tu także wskazówki, jak rozwiązywać różne problemy sieciowe, pisać skrypty do wykrywania zagrożeń sieci i stron internetowych, zabezpieczać urządzenia końcowe, pozyskiwać metadane i pisać skrypty kryptograficzne. Najważniejsze zagadnienia: skrypty automatyzujące procedury bezpieczeństwa i testy penetracyjne narzędzia programistyczne służące do zabezpieczania sieci automatyczna analiza serwerów wykrywanie podatności na ataki i analiza bezpieczeństwa praca z siecią Tor stosowanie narzędzi do analizy śledczej Python w sieci: najlepsza ochrona!

Big Data Analysis with Python. Combine Spark and Python to unlock the powers of parallel computing and machine learning

Ivan Marin, Ankit Shukla, Sarang VK

Processing big data in real time is challenging due to scalability, information inconsistency, and fault tolerance. Big Data Analysis with Python teaches you how to use tools that can control this data avalanche for you. With this book, you'll learn practical techniques to aggregate data into useful dimensions for posterior analysis, extract statistical measurements, and transform datasets into features for other systems.The book begins with an introduction to data manipulation in Python using pandas. You'll then get familiar with statistical analysis and plotting techniques. With multiple hands-on activities in store, you'll be able to analyze data that is distributed on several computers by using Dask. As you progress, you'll study how to aggregate data for plots when the entire data cannot be accommodated in memory. You'll also explore Hadoop (HDFS and YARN), which will help you tackle larger datasets. The book also covers Spark and explains how it interacts with other tools.By the end of this book, you'll be able to bootstrap your own Python environment, process large files, and manipulate data to generate statistics, metrics, and graphs.

Big Data on Kubernetes. A practical guide to building efficient and scalable data solutions

Neylson Crepalde

In today's data-driven world, organizations across different sectors need scalable and efficient solutions for processing large volumes of data. Kubernetes offers an open-source and cost-effective platform for deploying and managing big data tools and workloads, ensuring optimal resource utilization and minimizing operational overhead. If you want to master the art of building and deploying big data solutions using Kubernetes, then this book is for you.Written by an experienced data specialist, Big Data on Kubernetes takes you through the entire process of developing scalable and resilient data pipelines, with a focus on practical implementation. Starting with the basics, you’ll progress toward learning how to install Docker and run your first containerized applications. You’ll then explore Kubernetes architecture and understand its core components. This knowledge will pave the way for exploring a variety of essential tools for big data processing such as Apache Spark and Apache Airflow. You’ll also learn how to install and configure these tools on Kubernetes clusters. Throughout the book, you’ll gain hands-on experience building a complete big data stack on Kubernetes.By the end of this Kubernetes book, you’ll be equipped with the skills and knowledge you need to tackle real-world big data challenges with confidence.

Bioinformatics with Python Cookbook. Learn how to use modern Python bioinformatics libraries and applications to do cutting-edge research in computational biology - Second Edition

Tiago Antao

Bioinformatics is an active research field that uses a range of simple-to-advanced computations to extract valuable information from biological data.This book covers next-generation sequencing, genomics, metagenomics, population genetics, phylogenetics, and proteomics. You'll learn modern programming techniques to analyze large amounts of biological data. With the help of real-world examples, you'll convert, analyze, and visualize datasets using various Python tools and libraries.This book will help you get a better understanding of working with a Galaxy server, which is the most widely used bioinformatics web-based pipeline system. This updated edition also includes advanced next-generation sequencing filtering techniques. You'll also explore topics such as SNP discovery using statistical approaches under high-performance computing frameworks such as Dask and Spark.By the end of this book, you'll be able to use and implement modern programming techniques and frameworks to deal with the ever-increasing deluge of bioinformatics data.

Bioinformatics with Python Cookbook. Use modern Python libraries and applications to solve real-world computational biology problems - Third Edition

Tiago Antao

Bioinformatics is an active research field that uses a range of simple-to-advanced computations to extract valuable information from biological data, and this book will show you how to manage these tasks using Python.This updated third edition of the Bioinformatics with Python Cookbook begins with a quick overview of the various tools and libraries in the Python ecosystem that will help you convert, analyze, and visualize biological datasets. Next, you'll cover key techniques for next-generation sequencing, single-cell analysis, genomics, metagenomics, population genetics, phylogenetics, and proteomics with the help of real-world examples. You'll learn how to work with important pipeline systems, such as Galaxy servers and Snakemake, and understand the various modules in Python for functional and asynchronous programming. This book will also help you explore topics such as SNP discovery using statistical approaches under high-performance computing frameworks, including Dask and Spark. In addition to this, you’ll explore the application of machine learning algorithms in bioinformatics.By the end of this bioinformatics Python book, you'll be equipped with the knowledge you need to implement the latest programming techniques and frameworks, empowering you to deal with bioinformatics data on every scale.

Biostatistics with Python. Apply Python for biostatistics with hands-on biomedical and biotechnology projects

Darko Medin

This book leverages the author’s decade-long experience in biostatistics and data science to simplify the practical use of biostatistics with Python. The chapters show you how to clean and describe your data effectively, setting a solid foundation for accurate analysis and proficiency in biostatistical inference to help you draw meaningful conclusions from your data through hypothesis testing and effect size analysis.The book walks you through predictive modeling to harness the power of Python to create robust predictive analytics that can drive your research and professional projects forward. You'll explore clinical biostatistics, learn how to design studies, conduct survival analysis, and synthesize evidence from multiple studies with meta-analysis – skills that are crucial for making informed decisions based on comprehensive data reviews. The concluding chapters will enhance your ability to analyze biological variables, enabling you to perform detailed and accurate data analysis for biological research. This book's unique blend of biostatistics and Python helps you find practical solutions that make complex concepts easy to grasp and apply.By the end of this biostatistics book, you’ll have moved from theoretical knowledge to practical experience, allowing you to perform biostatistical analysis confidently and accurately.

Blender 3D By Example. A project-based guide to learning the latest Blender 3D, EEVEE rendering engine, and Grease Pencil - Second Edition

Oscar Baechler, Xury Greer

Blender is a powerful 3D creation package that supports every aspect of the 3D pipeline. With this book, you'll learn about modeling, rigging, animation, rendering, and much more with the help of some interesting projects.This practical guide, based on the Blender 2.83 LTS version, starts by helping you brush up on your basic Blender skills and getting you acquainted with the software toolset. You’ll use basic modeling tools to understand the simplest 3D workflow by customizing a Viking themed scene. You'll get a chance to see the 3D modeling process from start to finish by building a time machine based on provided concept art. You will design your first 2D character while exploring the capabilities of the new Grease Pencil tools. The book then guides you in creating a sleek modern kitchen scene using EEVEE, Blender’s new state-of-the-art rendering engine. As you advance, you'll explore a variety of 3D design techniques, such as sculpting, retopologizing, unwrapping, baking, painting, rigging, and animating to bring a baby dragon to life.By the end of this book, you'll have learned how to work with Blender to create impressive computer graphics, art, design, and architecture, and you'll be able to use robust Blender tools for your design projects and video games.

Building AI Applications with OpenAI APIs. Leverage ChatGPT, Whisper, and DALL-E APIs to build 10 innovative AI projects - Second Edition

Martin Yanev

Unlock the power of AI in your applications with ChatGPT with this practical guide that shows you how to seamlessly integrate OpenAI APIs into your projects, enabling you to navigate complex APIs and ensure seamless functionality with ease.This new edition is updated with key topics such as OpenAI Embeddings, which’ll help you understand the semantic relationships between words and phrases. You’ll find out how to use ChatGPT, Whisper, and DALL-E APIs through 10 AI projects using the latest OpenAI models, GPT-3.5, and GPT-4, with Visual Studio Code as the IDE. Within these projects, you’ll integrate ChatGPT with frameworks and tools such as Flask, Django, Microsoft Office APIs, and PyQt. You’ll get to grips with NLP tasks, build a ChatGPT clone, and create an AI code bug-fixing SaaS app. The chapters will also take you through speech recognition, text-to-speech capabilities, language translation, generating email replies, creating PowerPoint presentations, and fine-tuning ChatGPT, along with adding payment methods by integrating the ChatGPT API with Stripe.By the end of this book, you’ll be able to develop, deploy, and monetize your own groundbreaking applications by harnessing the full potential of ChatGPT APIs.

Building AI Intensive Python Applications. Create intelligent apps with LLMs and vector databases

Rachelle Palmer, Ben Perlmutter, Ashwin Gangadhar, Nicholas Larew, ...

The era of generative AI is upon us, and this book serves as a roadmap to harness its full potential. With its help, you’ll learn the core components of the AI stack: large language models (LLMs), vector databases, and Python frameworks, and see how these technologies work together to create intelligent applications.The chapters will help you discover best practices for data preparation, model selection, and fine-tuning, and teach you advanced techniques such as retrieval-augmented generation (RAG) to overcome common challenges, such as hallucinations and data leakage. You’ll get a solid understanding of vector databases, implement effective vector search strategies, refine models for accuracy, and optimize performance to achieve impactful results. You’ll also identify and address AI failures to ensure your applications deliver reliable and valuable results. By evaluating and improving the output of LLMs, you’ll be able to enhance their performance and relevance.By the end of this book, you’ll be well-equipped to build sophisticated AI applications that deliver real-world value.

Building Big Data Pipelines with Apache Beam. Use a single programming model for both batch and stream data processing

Jan Lukavský

Apache Beam is an open source unified programming model for implementing and executing data processing pipelines, including Extract, Transform, and Load (ETL), batch, and stream processing.This book will help you to confidently build data processing pipelines with Apache Beam. You’ll start with an overview of Apache Beam and understand how to use it to implement basic pipelines. You’ll also learn how to test and run the pipelines efficiently. As you progress, you’ll explore how to structure your code for reusability and also use various Domain Specific Languages (DSLs). Later chapters will show you how to use schemas and query your data using (streaming) SQL. Finally, you’ll understand advanced Apache Beam concepts, such as implementing your own I/O connectors.By the end of this book, you’ll have gained a deep understanding of the Apache Beam model and be able to apply it to solve problems.

Building Data Science Applications with FastAPI. Develop, manage, and deploy efficient machine learning applications with Python

François Voron

FastAPI is a web framework for building APIs with Python 3.6 and its later versions based on standard Python-type hints. With this book, you’ll be able to create fast and reliable data science API backends using practical examples.This book starts with the basics of the FastAPI framework and associated modern Python programming language concepts. You'll be taken through all the aspects of the framework, including its powerful dependency injection system and how you can use it to communicate with databases, implement authentication and integrate machine learning models. Later, you’ll cover best practices relating to testing and deployment to run a high-quality and robust application. You’ll also be introduced to the extensive ecosystem of Python data science packages. As you progress, you’ll learn how to build data science applications in Python using FastAPI. The book also demonstrates how to develop fast and efficient machine learning prediction backends and test them to achieve the best performance. Finally, you’ll see how to implement a real-time face detection system using WebSockets and a web browser as a client.By the end of this FastAPI book, you’ll have not only learned how to implement Python in data science projects but also how to maintain and design them to meet high programming standards with the help of FastAPI.

Building Modern Data Applications Using Databricks Lakehouse. Develop, optimize, and monitor data pipelines on Databricks

Will Girten

With so many tools to choose from in today’s data engineering development stack as well as operational complexity, this often overwhelms data engineers, causing them to spend less time gleaning value from their data and more time maintaining complex data pipelines. Guided by a lead specialist solutions architect at Databricks with 10+ years of experience in data and AI, this book shows you how the Delta Live Tables framework simplifies data pipeline development by allowing you to focus on defining input data sources, transformation logic, and output table destinations.This book gives you an overview of the Delta Lake format, the Databricks Data Intelligence Platform, and the Delta Live Tables framework. It teaches you how to apply data transformations by implementing the Databricks medallion architecture and continuously monitor the data quality of your pipelines. You’ll learn how to handle incoming data using the Databricks Auto Loader feature and automate real-time data processing using Databricks workflows. You’ll master how to recover from runtime errors automatically.By the end of this book, you’ll be able to build a real-time data pipeline from scratch using Delta Live Tables, leverage CI/CD tools to deploy data pipeline changes automatically across deployment environments, and monitor, control, and optimize cloud costs.

Building Python Web APIs with FastAPI. A fast-paced guide to building high-performance, robust web APIs with very little boilerplate code

Abdulazeez Abdulazeez Adeshina

RESTful web services are commonly used to create APIs for web-based applications owing to their light weight and high scalability. This book will show you how FastAPI, a high-performance web framework for building RESTful APIs in Python, allows you to build robust web APIs that are simple and intuitive and makes it easy to build quickly with very little boilerplate code.This book will help you set up a FastAPI application in no time and show you how to use FastAPI to build a REST API that receives and responds to user requests. You’ll go on to learn how to handle routing and authentication while working with databases in a FastAPI application. The book walks you through the four key areas: building and using routes for create, read, update, and delete (CRUD) operations; connecting the application to SQL and NoSQL databases; securing the application built; and deploying your application locally or to a cloud environment.By the end of this book, you’ll have developed a solid understanding of the FastAPI framework and be able to build and deploy robust REST APIs.

Building Serverless Applications with Python. Develop fast, scalable, and cost-effective web applications that are always available

Jalem Raj Rohit

Serverless architectures allow you to build and run applications and services without having to manage the infrastructure. Many companies have adopted this architecture to save cost and improve scalability. This book will help you design serverless architectures for your applications with AWS and Python.The book is divided into three modules. The first module explains the fundamentals of serverless architecture and how AWS lambda functions work. In the next module, you will learn to build, release, and deploy your application to production. You will also learn to log and test your application. In the third module, we will take you through advanced topics such as building a serverless API for your application. You will also learn to troubleshoot and monitor your app and master AWS lambda programming concepts with API references. Moving on, you will also learn how to scale up serverless applications and handle distributed serverless systems in production.By the end of the book, you will be equipped with the knowledge required to build scalable and cost-efficient Python applications with a serverless framework.

Building Serverless Python Web Services with Zappa. Build and deploy serverless applications on AWS using Zappa

Abdulwahid Abdulhaque Barguzar

Serverless applications are becoming very popular these days, not just because they save developers the trouble of managing the servers, but also because they provide several other benefits such as cutting heavy costs and improving the overall performance of the application.This book will help you build serverless applications in a quick and efficient way. We begin with an introduction to AWS and the API gateway, the environment for serverless development, and Zappa. We then look at building, testing, and deploying apps in AWS with three different frameworks--Flask, Django, and Pyramid. Setting up a custom domain along with SSL certificates and configuring them with Zappa is also covered. A few advanced Zappa settings are also covered along with securing Zappa with AWS VPC.By the end of the book you will have mastered using three frameworks to build robust and cost-efficient serverless apps in Python.

Clean Code in Python. Develop maintainable and efficient code - Second Edition

Mariano Anaya

Experienced professionals in every field face several instances of disorganization, poor readability, and testability due to unstructured code.With updated code and revised content aligned to the new features of Python 3.9, this second edition of Clean Code in Python will provide you with all the tools you need to overcome these obstacles and manage your projects successfully.The book begins by describing the basic elements of writing clean code and how it plays a key role in Python programming. You will learn about writing efficient and readable code using the Python standard library and best practices for software design.The book discusses object-oriented programming in Python and shows you how to use objects with descriptors and generators. It will also show you the design principles of software testing and how to resolve problems by implementing software design patterns in your code. In the concluding chapter, we break down a monolithic application into a microservices-based one starting from the code as the basis for a solid platform.By the end of this clean code book, you will be proficient in applying industry-approved coding practices to design clean, sustainable, and readable real-world Python code.