Creating Singer Target to Send Data to Web Endpoint

30. June 2023 13:33 by Jay Grossman in   //  Tags: , ,   //   Comments (0)
Please NoteIf you are new to Singer, you may want to check out my last post Creating Singer Tap to Capture Ebay Completed Items. It provides a high level background of the specification and how taps & targets work together.   The Challenge Last week I created a walk through of Creating Singer Tap to Capture Ebay Completed Items. While it's great to capture data, it's not overly useful without persisting the data to a target destination. There are some useful targets p... [More]

Creating Singer Tap to Capture Ebay Completed Items

19. June 2023 20:57 by Jay Grossman in   //  Tags: , , ,   //   Comments (0)
What is Singer The Singer specification bills itself as "the open-source standard for writing scripts that move data". PipelineWise and Meltano are popular open source platforms that use the Singer specification to accommodate ingest and replication of data from various sources to various destinations. Singer describes how data extraction scripts—called "taps" —and data loading scripts—called "targets"— should communicate, allowing them to be used in any combinatio... [More]

Dagster with Python, Singer, and Meltano

4. April 2023 12:16 by Jay Grossman in   //  Tags: , , , , , ,   //   Comments (0)
I have been a fan of Dagster for data orchestration for a little while and wanted to share some of the basics. There are a lot of cool things I like about it (compared to airflow and other schedulers): It is Declarative (via their Software-Defined Asset object). I personally like tools and frameworks that allow me to declare a desired end state (Terraform, dbt, Puppet, Ansible, etc.) vs. frameworks that have me build a bunch of imperative tasks that get daisy chained together.    I... [More]

About the author

Jay Grossman

techie / entrepreneur that enjoys:
 1) my kids + awesome wife
 2) building software projects/products
 3) digging for gold in data sets
 4) my various day jobs
 5) rooting for my Boston sports teams:
    New England PatriotsBoston Red SoxBoston CelticsBoston Bruins

Month List