How To Setup A Batch Data Pipeline For CSV Files

Popular posts from this blog

Data Lake Icon: Visual Reference

Image
Data Lake Icon If you were looking for some data lake icons or data lake logos, our creative team created a few for Openbridge. Data Lake Icons In terms of consistent visual representations of a data lake, we were not happy with what was available. Rather than create a unique AWS data lake icon or an Azure data lake icon, we wanted these to apply generally across all platforms. As a result, we created a set of icons that reflect a data lake conceptually. We included a modified icon that reflects our data lake formation process: data lake logo Data Lake Icons Use Cases The following are some examples of how the icons are used Openbridge : Using the icon in animation in a data lake introduction visual block Using the icon in the “ what is a data lake ” page header Using the icon in the “ why data lake ?” graphic Using the task icon to describe formation & architecting data lakes Looking for data lake providers or a data lake roadmap? If you are looking for icons
Read more

Snowflake CREATE Warehouse

Image
How to script a consistent and automated “create warehouse snowflake” environment Prior to undertaking data loading, Snowflake should be configured with some baselines. In this post, we undertake some “lite” Snowflake data warehouse training using the WORKSHEET feature. This is an easy-to-understand starting point in anyone’s journey to better learn Snowflake data warehouse setup and operations. If you are wondering why use Snowflake data warehouse , check out that link for the benefits of the platform. For now, we are assuming you are sold and are ready to give it a try. Let's get started! Step 1: Create a Snowflake account If you do not already have an account, Snowflake offers a free trial . Sign up for an account. If you have an account already, log into it. Step 2: Locate the WORKSHEET Feature In Your Snowflake Account If you log in via a web browser, click the WORKSHEET icon at the top of the page. Important: Ensure that the role for the session is set to ACCOUNTAD
Read more

PrestoDB vs PrestoSQL & the new Presto Foundation

Image
PrestoDB Last year we posted an introduction article on Presto . So what is new in the Presto world since then? A ton! PrestoDB moves to the Linux Foundation In September 2019, the official PrestoDB Foundation was started by Facebook, Uber, Twitter, and Alibaba. A formal, official foundation is what was needed for the Presto ecosystem to prosper. The formation and transition to a formal foundation under the Linux Foundation’s auspices was a significant first step to deal with confusion in the community. Kudos to Facebook, Uber, Twitter, and others in making this a reality. Having open, shared, and community-driven organization is critical to future success Presto. You can read more about these principles and roadmaps  here . Revisiting PrestoDB vs. PrestoSQL Why is a formal, independent foundation necessary? Presto Foundation established a set of much-needed guiding principles for the community. In the post last year, we highlighted some confusion about the two principle Pr
Read more