AWS Data Pipeline — A Introduction To A Platform For Solving Complex Data Pipeline Headaches

Popular posts from this blog

How To Create Data Lake In Azure

Image
How to create Azure Data Lake Storage Gen2 destinations Three simple steps show you how to create Azure Data Lake Storage Gen2 data destinations for insights and analytics Configuring an Azure Data Lake destination is quick and easy. In this post, we walk you through the steps to get a data lake configured in your Azure account. This guide assumes you have an Azure account and understand why a data lake is a good fit for your use case. If you ask yourself, “ what is a data lake ?” or struggling with a “ data lake vs. data warehouse ” question, check out those links. Ready to get started? Let’s go… Three Steps To Setting Up Azure Three primary steps align to specific resources within Azure; A Gen 2 storage account, a data lake container, and access credentials within Azure. These three elements are needed by other systems to establish a connection to your Azure Data Lake destination securely: Storage Account Name Container Name Connection strin...
Read more

Snowflake CREATE Warehouse

Image
How to script a consistent and automated “create warehouse snowflake” environment Prior to undertaking data loading, Snowflake should be configured with some baselines. In this post, we undertake some “lite” Snowflake data warehouse training using the WORKSHEET feature. This is an easy-to-understand starting point in anyone’s journey to better learn Snowflake data warehouse setup and operations. If you are wondering why use Snowflake data warehouse , check out that link for the benefits of the platform. For now, we are assuming you are sold and are ready to give it a try. Let's get started! Step 1: Create a Snowflake account If you do not already have an account, Snowflake offers a free trial . Sign up for an account. If you have an account already, log into it. Step 2: Locate the WORKSHEET Feature In Your Snowflake Account If you log in via a web browser, click the WORKSHEET icon at the top of the page. Important: Ensure that the role for the session...
Read more

Data Lake Icon: Visual Reference

Image
Data Lake Icon If you were looking for some data lake icons or data lake logos, our creative team created a few for Openbridge. Data Lake Icons In terms of consistent visual representations of a data lake, we were not happy with what was available. Rather than create a unique AWS data lake icon or an Azure data lake icon, we wanted these to apply generally across all platforms. As a result, we created a set of icons that reflect a data lake conceptually. We included a modified icon that reflects our data lake formation process: data lake logo Data Lake Icons Use Cases The following are some examples of how the icons are used Openbridge : Using the icon in animation in a data lake introduction visual block Using the icon in the “ what is a data lake ” page header Using the icon in the “ why data lake ?” graphic Using the task icon to describe formation & architecting data lakes Looking for data lake providers or a data lake...
Read more