Apache Airflow Best Practices. A practical guide to orchestrating data workflow with Apache Airflow

Dylan Intorf, Dylan Storey, Kendrick van Doorn

Data professionals face the monumental task of managing complex data pipelines, orchestrating workflows across diverse systems, and ensuring scalable, reliable data processing. This definitive guide to mastering Apache Airflow, written by experts in engineering, data strategy, and problem-solving across tech, financial, and life sciences industries, is your key to overcoming these challenges. It covers everything from the basics of Airflow and its core components to advanced topics such as custom plugin development, multi-tenancy, and cloud deployment.Starting with an introduction to data orchestration and the significant updates in Apache Airflow 2.0, this book takes you through the essentials of DAG authoring, managing Airflow components, and connecting to external data sources. Through real-world use cases, you’ll gain practical insights into implementing ETL pipelines and machine learning workflows in your environment. You’ll also learn how to deploy Airflow in cloud environments, tackle operational considerations for scaling, and apply best practices for CI/CD and monitoring. By the end of this book, you’ll be proficient in operating and using Apache Airflow, authoring high-quality workflows in Python for your specific use cases, and making informed decisions crucial for production-ready implementation.

Apache Camel Developer's Cookbook. For Apache Camel developers, this is the book you'll always want to have handy. It's stuffed full of great recipes that are designed for quick practical application. Expands your Apache Camel abilities immediately

Scott Cranton, Jakub Korab

Apache Camel is a de-facto standard for developing integrations in Java, and is based on well-understood Enterprise Integration Patterns. It is used within many commercial and open source integration products. Camel makes common integration tasks easy while still providing the developer with the means to customize the framework when the situation demands it. Tasks such as protocol mediation, message routing and transformation, and auditing are common usages of Camel. Apache Camel Developer's Cookbook provides hundreds of best practice tips for using Apache Camel in a format that helps you build your Camel projects. Each tip or recipe provides you with the most important steps to perform along with a summary of how it works, with references to further reading if you need more information. This book is intended to be a reliable information source that is quicker to use than an Internet search. Apache Camel Developer's Cookbook is a quick lookup guide that can also be read from cover to cover if you want to get a sense of the full power of Apache Camel. This book provides coverage of the full lifecycle of creating Apache Camel-based integration projects, including the structure of your Camel code and using the most common Enterprise Integration patterns. Patterns like Split/Join and Aggregation are covered in depth in this book. Throughout this book, you will be learning steps to transform your data. You will also learn how to perform unit and integration testing of your code using Camel's extensive testing framework, and also strategies for debugging and monitoring your code. Advanced topics like Error Handling, Parallel Processing, Transactions, and Security will also be covered in this book. This book provides you with practical tips on using Apache Camel based on years of hands-on experience from hundreds of integration projects.

Apache Cassandra Essentials. Create your own massively scalable Cassandra database with highly responsive database queries

Nitin Padalia

Apache Cassandra Essentials takes you step-by-step from from the basics of installation to advanced installation options and database design techniques. It gives you all the information you need to effectively design a well distributed and high performance database. You’ll get to know about the steps that are performed by a Cassandra node when you execute a read/write query, which is essential to properly maintain of a Cassandra cluster and to debug any issues. Next, you’ll discover how to integrate a Cassandra driver in your applications and perform read/write operations. Finally, you’ll learn about the various tools provided by Cassandra for serviceability aspects such as logging, metrics, backup, and recovery.

Apache Flume: Distributed Log Collection for Hadoop. Design and implement a series of Flume agents to send streamed data into Hadoop

Steve Hoffman, Steven Hoffman

Apache Flume: Distributed Log Collection for Hadoop. If your role includes moving datasets into Hadoop, this book will help you do it more efficiently using Apache Flume. From installation to customization, it's a complete step-by-step guide on making the service work for you

Steve Hoffman, Steven Hoffman, Kevin A. McGrail

Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. Its main goal is to deliver data from applications to Apache Hadoop's HDFS. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with many failover and recovery mechanisms.Apache Flume: Distributed Log Collection for Hadoop covers problems with HDFS and streaming data/logs, and how Flume can resolve these problems. This book explains the generalized architecture of Flume, which includes moving data to/from databases, NO-SQL-ish data stores, as well as optimizing performance. This book includes real-world scenarios on Flume implementation.Apache Flume: Distributed Log Collection for Hadoop starts with an architectural overview of Flume and then discusses each component in detail. It guides you through the complete installation process and compilation of Flume.It will give you a heads-up on how to use channels and channel selectors. For each architectural component (Sources, Channels, Sinks, Channel Processors, Sink Groups, and so on) the various implementations will be covered in detail along with configuration options. You can use it to customize Flume to your specific needs. There are pointers given on writing custom implementations as well that would help you learn and implement them.By the end, you should be able to construct a series of Flume agents to transport your streaming data and logs from your systems into Hadoop in near real time.

Apache Gold. A Story of the Strange Southwest

Joseph A. Altsheler

Apache Gold, A Story of the Strange Southwest is a rip-roaring tale of adventure set on the Arizona frontiers of the American Old West written by Joseph Alexander Altsheler (April 29, 1862 June 5, 1919). He was an American newspaper reporter, editor and author of popular juvenile historical fiction. An abiding classic of western literature, our hero in this tale is Charles Wayne, a young but strong and sharp lad who seeks adventure in the southwesterly desert frontier of Arizona. Charles has terrifying encounters with wild beasts and Indians when he searches for the lost treasure of the Spaniards. Joseph A. Altsheler describes the vast open frontier evocatively, placing the reader in a time when equal measures of freedom and danger were abundant. Throughout Mr. Waynes traversals, were reminded of how difficult it was to survive let alone thrive in the Old West. The beauty of the unforgiving land forms a vibrant backdrop to the scrapes and challenges our heroes must face.

Apache Hadoop 3 Quick Start Guide. Learn about big data processing and analytics

Hrishikesh Vijay Karambelkar

Apache Hadoop is a widely used distributed data platform. It enables large datasets to be efficiently processed instead of using one large computer to store and process the data. This book will get you started with the Hadoop ecosystem, and introduce you to the main technical topics, including MapReduce, YARN, and HDFS.The book begins with an overview of big data and Apache Hadoop. Then, you will set up a pseudo Hadoop development environment and a multi-node enterprise Hadoop cluster. You will see how the parallel programming paradigm, such as MapReduce, can solve many complex data processing problems.The book also covers the important aspects of the big data software development lifecycle, including quality assurance and control, performance, administration, and monitoring. You will then learn about the Hadoop ecosystem, and tools such as Kafka, Sqoop, Flume, Pig, Hive, and HBase. Finally, you will look at advanced topics, including real time streaming using Apache Storm, and data analytics using Apache Spark. By the end of the book, you will be well versed with different configurations of the Hadoop 3 cluster.

Apache Hive Cookbook

Hanish Bansal, Shrey Mehrotra, Saurabh Chauhan

Hive was developed by Facebook and later open sourced in Apache community. Hive provides SQL like interface to run queries on Big Data frameworks. Hive provides SQL like syntax also called as HiveQL that includes all SQL capabilities like analytical functions which are the need of the hour in today’s Big Data world.This book provides you easy installation steps with different types of metastores supported by Hive. This book has simple and easy to learn recipes for configuring Hive clients and services. You would also learn different Hive optimizations including Partitions and Bucketing. The book also covers the source code explanation of latest Hive version.Hive Query Language is being used by other frameworks including spark. Towards the end you will cover integration of Hive with these frameworks.

Apache Hive Essentials. Essential techniques to help you process, and get unique insights from, big data - Second Edition

Dayong Du

In this book, we prepare you for your journey into big data by frstly introducing you to backgrounds in the big data domain, alongwith the process of setting up and getting familiar with your Hive working environment. Next, the book guides you through discovering and transforming the values of big data with the help of examples. It also hones your skills in using the Hive language in an effcient manner. Toward the end, the book focuses on advanced topics, such as performance, security, and extensions in Hive, which will guide you on exciting adventures on this worthwhile big data journey. By the end of the book, you will be familiar with Hive and able to work effeciently to find solutions to big data problems

Apache Hive Essentials. Immerse yourself on a fantastic journey to discover the attributes of big data by using Hive

Dayong Du

Apache Ignite Quick Start Guide. Distributed data caching and processing made easy

Sujoy Acharya

Apache Ignite is a distributed in-memory platform designed to scale and process large volume of data. It can be integrated with microservices as well as monolithic systems, and can be used as a scalable, highly available and performant deployment platform for microservices. This book will teach you to use Apache Ignite for building a high-performance, scalable, highly available system architecture with data integrity.The book takes you through the basics of Apache Ignite and in-memory technologies. You will learn about installation and clustering Ignite nodes, caching topologies, and various caching strategies, such as cache aside, read and write through, and write behind. Next, you will delve into detailed aspects of Ignite’s data grid: web session clustering and querying data.You will learn how to process large volumes of data using compute grid and Ignite’s map-reduce and executor service. You will learn about the memory architecture of Apache Ignite and monitoring memory and caches. You will use Ignite for complex event processing, event streaming, and the time-series predictions of opportunities and threats. Additionally, you will go through off-heap and on-heap caching, swapping, and native and Spring framework integration with Apache Ignite.By the end of this book, you will be confident with all the features of Apache Ignite 2.x that can be used to build a high-performance system architecture.

Apache Kafka 1.0 Cookbook. Over 100 practical recipes on using distributed enterprise messaging to handle real-time data

Raúl Estrada

Apache Kafka provides a unified, high-throughput, low-latency platform to handle real-time data feeds. This book will show you how to use Kafka efficiently, and contains practical solutions to the common problems that developers and administrators usually face while working with it. This practical guide contains easy-to-follow recipes to help you set up, configure, and use Apache Kafka in the best possible manner. You will use Apache Kafka Consumers and Producers to build effective real-time streaming applications. The book covers the recently released Kafka version 1.0, the Confluent Platform and Kafka Streams. The programming aspect covered in the book will teach you how to perform important tasks such as message validation, enrichment and composition.Recipes focusing on optimizing the performance of your Kafka cluster, and integrate Kafka with a variety of third-party tools such as Apache Hadoop, Apache Spark, and Elasticsearch will help ease your day to day collaboration with Kafka greatly. Finally, we cover tasks related to monitoring and securing your Apache Kafka cluster using tools such as Ganglia and Graphite.If you're looking to become the go-to person in your organization when it comes to working with Apache Kafka, this book is the only resource you need to have.

Apache Kafka Quick Start Guide. Leverage Apache Kafka 2.0 to simplify real-time data processing for distributed applications

Raúl Estrada

Apache Kafka is a great open source platform for handling your real-time data pipeline to ensure high-speed filtering and pattern matching on the ?y. In this book, you will learn how to use Apache Kafka for efficient processing of distributed applications and will get familiar with solving everyday problems in fast data and processing pipelines.This book focuses on programming rather than the configuration management of Kafka clusters or DevOps. It starts off with the installation and setting up the development environment, before quickly moving on to performing fundamental messaging operations such as validation and enrichment.Here you will learn about message composition with pure Kafka API and Kafka Streams. You will look into the transformation of messages in different formats, such asext, binary, XML, JSON, and AVRO. Next, you will learn how to expose the schemas contained in Kafka with the Schema Registry. You will then learn how to work with all relevant connectors with Kafka Connect. While working with Kafka Streams, you will perform various interesting operations on streams, such as windowing, joins, and aggregations. Finally, through KSQL, you will learn how to retrieve, insert, modify, and delete data streams, and how to manipulate watermarks and windows.

Apache Karaf Cookbook. Over 60 recipes to help you get the most out of your Apache Karaf deployments

Jamie Goodyear, Johan Edstorm, Achim Nierbeck, Heath J Kesler

Apache Mahout Clustering Designs. Explore clustering algorithms used with Apache Mahout

Dragan Milcevski, Ashish Gupta, Ashish Gupta

Apache Mahout Essentials. Implement top-notch machine learning algorithms for classification, clustering, and recommendations with Apache Mahout

Jayani Withanawasam

Apache Maven Cookbook. Over 90 hands-on recipes to successfully build and automate development life cycle tasks following Maven conventions and best practices

Raghuram Bharathan, SRIRANGAN .

Apache Mesos Cookbook. Efficiently handle and manage tasks in a distributed environment

David Blomquist, Tomasz Janiszewski

Apache Mesos is open source cluster sharing and management software. Deploying and managing scalable applications in large-scale clustered environments can be difficult, but Apache Mesos makes it easier with efficient resource isolation and sharing across application frameworks.The goal of this book is to guide you through the practical implementation of the Mesos core along with a number of Mesos supported frameworks. You will begin by installing Mesos and then learn how to configure clusters and maintain them. You will also see how to deploy a cluster in a production environment with high availability using Zookeeper.Next, you will get to grips with using Mesos, Marathon, and Docker to build and deploy a PaaS. You will see how to schedule jobs with Chronos. We’ll demonstrate how to integrate Mesos with big data frameworks such as Spark, Hadoop, and Storm. Practical solutions backed with clear examples will also show you how to deploy elastic big data jobs. You will find out how to deploy a scalable continuous integration and delivery system on Mesos with Jenkins. Finally, you will configure and deploy a highly scalable distributed search engine with ElasticSearch.Throughout the course of this book, you will get to know tips and tricks along with best practices to follow when working with Mesos.

Apache OfBiz Cookbook. Over 60 simple but incredibly effective recipes for taking control of OFBiz

Ruth Hoffman, Brian Fitzpatrick

Apache Open For Business (OFBiz) is an enterprise resource planning (ERP) system that provides a common data model and an extensive set of business processes. But without proper guidance on developing performance-critical applications, it is easy to make the wrong design and technology decisions. The power and promise of Apache OFBiz is comprehensively revealed in a collection of self-contained, quick, practical recipes in this Cookbook.This book covers a range of topics from initial system setup to web application and HTML page creation, Java development, and data maintenance tasks. Focusing on a series of the most commonly performed OFBiz tasks, it provides clear, cogent, and easy-to-follow instructions designed to make the most of your OFBiz experience.Let this book be your guide to enhancing your OFBiz productivity by saving you valuable time. Written specifically to give clear and straightforward answers to the most commonly asked OFBiz questions, this compendium of OFBiz recipes will show you everything you need to know to get things done in OFBiz.Whether you are new to OFBiz or an old pro, you are sure to find many useful hints and handy tips here. Topics range from getting started to configuration and system setup, security and database management through the final stages of developing and testing new OFBiz applications.

Apache Oozie Essentials. Unleash the power of Apache Oozie to create and manage your big data and machine learning pipelines in one go

Jagat Jasjit Singh

As more and more organizations are discovering the use of big data analytics, interest in platforms that provide storage, computation, and analytic capabilities is booming exponentially. This calls for data management. Hadoop caters to this need. Oozie fulfils this necessity for a scheduler for a Hadoop job by acting as a cron to better analyze data. Apache Oozie Essentials starts off with the basics right from installing and configuring Oozie from source code on your Hadoop cluster to managing your complex clusters. You will learn how to create data ingestion and machine learning workflows.This book is sprinkled with the examples and exercises to help you take your big data learning to the next level. You will discover how to write workflows to run your MapReduce, Pig ,Hive, and Sqoop scripts and schedule them to run at a specific time or for a specific business requirement using a coordinator. This book has engaging real-life exercises and examples to get you in the thick of things. Lastly, you’ll get a grip of how to embed Spark jobs, which can be used to run your machine learning models on Hadoop.By the end of the book, you will have a good knowledge of Apache Oozie. You will be capable of using Oozie to handle large Hadoop workflows and even improve the availability of your Hadoop environment.

Apache. Receptury. Wydanie II

Rich Bowen, Ken Coar

Czy wiesz, jaki serwer HTTP jest najpopularniejszy w sieci? Właśnie tak, jest to Apache! W lipcu 2008 roku jego udział w rynku wynosił blisko 50% (według Netcraft). Historia tego serwera sięga roku 1995, kiedy ukazała się jego pierwsza oficjalna wersja, oznaczona numerem 0.6.2. Cechy, które zadecydowały o sukcesie tego rozwiązania, to bezpieczeństwo, skalowalność, wielowątkowość i obsługa różnorodnych języków skryptowych. Dzięki książce "Apache. Receptury" zapoznasz się z gotowymi przepisami na rozwiązanie ciekawych, specyficznych oraz intrygujących problemów. Nauczysz się instalować serwer z różnych źródeł oraz na różnych platformach. Dowiesz się, w jaki sposób zwiększyć jego bezpieczeństwo, jak uruchomić serwery wirtualne oraz poprawić wydajność Apache. Autorzy książki pokażą Ci, jak uruchomić obsługę języków skryptowych, tak aby serwowane strony stały się dynamiczne. Cała wiedza zostanie przedstawiona w sprawdzony w tej serii sposób: problem - rozwiązanie - analiza. Sposoby instalacji serwera Apache Dodawanie funkcjonalności dzięki modułom Możliwości rejestracji zdarzeń Konfiguracja serwerów wirtualnych Wykorzystanie aliasów, przekierowań oraz przepisań (mod_rewrite) Zarządzanie dostępem do serwowanych zasobów Bezpieczeństwo serwera Apache Wykorzystanie szyfrowanej transmisji - protokół SSL Zapewnienie wydajności Wykorzystanie języków skryptowych Oto książka z najlepszymi przepisami na Apache!

Apache Roller 4.0 - Beginner's Guide. A comprehensive, step-by-step guide on how to set up, customize, and market your blog using Apache Roller

Alfonso V. Romero, Brian Fitzpatrick, Alfonso Vidal Romero

Apache Roller enables you to build a fully-featured, multi-user blog server apt for all kinds of blogging sites. It is an ideal tool to create your own blogging network with unlimited users and blogs, forums, photo galleries, and more! While it is exciting to have a list of interesting features it can offer you, it might be a little difficult to get started with it by your self.This book will teach you how to get started with Apache Roller and make the most of all its features using step-by-step, detailed instructions. You will learn how to establish your internet presence with an Apache Roller blog and use the latest web tools to enhance your posts and attract visitors. You will also learn how to promote your blog on popular social bookmarking services and customize it to suit your need.This hands-on and practical book introduces you to Apache Roller. Starting off with the configuration and installation of your own blog, you'll then quickly learn how to add interesting content to your blog with the help of plenty of examples. You'll also learn how to change your blog's visual appearance with the help of Roller themes and templates and how to create a community of blogs for you and your colleagues or friends in your Apache Roller blog server. The book also looks at ways you can manage your community, and keep your site safe and secure, ensuring that it is a spam-free, enjoyable community for your users.

Apache Solr for Indexing Data. Enhance your Solr indexing experience with advanced techniques and the built-in functionalities available in Apache Solr

Anshul Johri, Sachin Handiekar

Apache Solr is a widely used, open source enterprise search server that delivers powerful indexing and searching features. These features help fetch relevant information from various sources and documentation. Solr also combines with other open source tools such as Apache Tika and Apache Nutch to provide more powerful features.This fast-paced guide starts by helping you set up Solr and get acquainted with its basic building blocks, to give you a better understanding of Solr indexing. You’ll quickly move on to indexing text and boosting the indexing time. Next, you’ll focus on basic indexing techniques, various index handlers designed to modify documents, and indexing a structured data source through Data Import Handler.Moving on, you will learn techniques to perform real-time indexing and atomic updates, as well as more advanced indexing techniques such as de-duplication. Later on, we’ll help you set up a cluster of Solr servers that combine fault tolerance and high availability. You will also gain insights into working scenarios of different aspects of Solr and how to use Solr with e-commerce data.By the end of the book, you will be competent and confident working with indexing and will have a good knowledge base to efficiently program elements.

Apache Solr High Performance. In setting up Apache Solr, you’ll want to ensure it’s achieving optimum search results with maximum efficiency. This book shows you just how to achieve that with a comprehensive tutorial including troubleshooting

Surendra Mohan