How To Setup A Batch Data Pipeline For CSV Files

Popular posts from this blog

Data Lake Icon: Visual Reference

Image
Data Lake Icon If you were looking for some data lake icons or data lake logos, our creative team created a few for Openbridge. Data Lake Icons In terms of consistent visual representations of a data lake, we were not happy with what was available. Rather than create a unique AWS data lake icon or an Azure data lake icon, we wanted these to apply generally across all platforms. As a result, we created a set of icons that reflect a data lake conceptually. We included a modified icon that reflects our data lake formation process: data lake logo Data Lake Icons Use Cases The following are some examples of how the icons are used Openbridge : Using the icon in animation in a data lake introduction visual block Using the icon in the “ what is a data lake ” page header Using the icon in the “ why data lake ?” graphic Using the task icon to describe formation & architecting data lakes Looking for data lake providers or a data lake roadmap? If you are looking for icons
Read more

Why Timeszones Cause Amazon Seller Central Confusion

Image
Timezones are a confusing topic when it comes to a global marketplace like Amazon. For example, a customer in New York places an order on March 11, 2021, at 1 AM (EST), but it is March 10, 2021, at 10 PM (PST) in Seattle. What when did the order occur? March 11th or March 10th? Another wrinkle is that Amazon defaults to using a Coordinated Universal Time (UTC) standard in its systems. In our example above, the timestamp of the order would be March 11th, 2021, at 8 AM! Seller Central User Interface So, where does a chunk of the confusion arise? Amazon Seller Central. The Seller Central interface displays information in the Pacific timezone (PST). However, Amazon stores information in its systems using UTC. Per Amazon: “Amazon Marketplace Web Service (Amazon MWS) formats date- and time-related data similar to the RFC 3339 standard, which defines a subset profile of ISO 8601 for use in Internet protocols and standards. This section will clarify just how you should format and use d
Read more

PrestoDB vs PrestoSQL & the new Presto Foundation

Image
PrestoDB Last year we posted an introduction article on Presto . So what is new in the Presto world since then? A ton! PrestoDB moves to the Linux Foundation In September 2019, the official PrestoDB Foundation was started by Facebook, Uber, Twitter, and Alibaba. A formal, official foundation is what was needed for the Presto ecosystem to prosper. The formation and transition to a formal foundation under the Linux Foundation’s auspices was a significant first step to deal with confusion in the community. Kudos to Facebook, Uber, Twitter, and others in making this a reality. Having open, shared, and community-driven organization is critical to future success Presto. You can read more about these principles and roadmaps  here . Revisiting PrestoDB vs. PrestoSQL Why is a formal, independent foundation necessary? Presto Foundation established a set of much-needed guiding principles for the community. In the post last year, we highlighted some confusion about the two principle Pr
Read more