Data pipelining is the act of moving data from one location to another and is one of the most critical operations in data infrastructure. Data pipelines are a superset of Extract, Transform, Load (ETL) and modern ELT. Additional data pipeline technologies include reverse ETL and integration and workflow automation vendors. While data pipelining isn’t new, as more companies have become data-driven, moved to the cloud, and adopt cloud data warehouses, we’ve seen the emergence of numerous new data pipelining solutions.

Data pipelines help teams get more leverage out of their data for business decisions and product experiences. With the…

Last Wednesday, I hosted Pavi Sandhu to discuss the importance of building and maintaining good developer documentation. Pavi has more than 20 years of experience leading documentation and user education initiatives at companies such as Salesforce, Amazon, Mesosphere, and MongoDB. Pavi co-founded Salesforce’s learning platform, Trailhead, which now has more than two million users. Below we will discuss some key takeaways from the discussion.

Technical marketing vs. documentation. Technical marketing focuses on informational content like data sheets and white papers that are used to encourage product adoption. Documentation comes into play after someone decides to use the solution. It…

Demos can be problematic. Even the best teams can suffer from bad demos like when Bill Gates presented Windows 98 and saw the “Blue Screen of Death” to Steve Jobs being unable to connect the iPhone to Wi-Fi. No wonder demo anxiety is real. Sadly, demos often fail because of networking and infrastructure challenges. It can be a lot of work to even create a demo. Go-to-market (GTM) teams rely on backend-end teams like graphic designers and developers to build demos and custom experiences, but this process can take time and be costly. …

On Wednesday, March 31st at 9:00 AM PT, Redpoint Office Hours will host Pavi Sandhu. Pavi has more than 20 years of experience leading documentation and user education initiatives, at companies such as Salesforce, Amazon, Mesosphere, and MongoDB. Pavi co-founded Salesforce’s learning platform, Trailhead, which now has more than two million users. He’s currently the founder and CEO of Therabot, a mental healthcare startup in San Francisco. He’s also the author of The MathML Handbook and has a PhD in physics from Boston University.

Redpoint investor, Astasia Myers, who has invested in companies such as LaunchDarkly, StackRox, Dremio, Gremlin…

Over the next few years, we expect developer portals to become a key piece of infrastructure for developers. A developer portal unifies all infrastructure tooling, services, and documentation with a single consistent UI. We highlighted developer portals in our “Introducing Redpoint’s SRE Landscape,” and since then the category has blossomed. Below we discuss the trends catalyzing developer portal adoption, explain its benefits, and categorize 18 offerings in the space.

Historically engineers used service catalogs like ServiceNow to identify the resources available to them. However, these solutions have been challenged by the movement to microservices, the cloud, and Bring Your…

Data infrastructure has gone through an incredible evolution over the past three years. We have moved from Extract, Transform, Load (ETL) to ELT, where raw data is copied from the source system loaded into a data warehouse or data lake and then transformed. Now teams are adopting yet another new approach, called “reverse ETL,” the process of moving data from a data warehouse into third party systems to make data operational. The emergence of reverse ETL solutions is a useful component of the stack to get better leverage out of data.

As teams rearchitect from ETL to ELT, the…

Data is becoming more critical for running a highly functional business from ad hoc data analysis to Business Intelligence (BI) to Machine Learning (ML). Data quality ensures data is fit for consumption and meets the needs of data consumers. Historically individuals did not have data quality tools and had to manually identify data issues. Over the past two years, we have seen the emergence of numerous data quality solutions and believe it is a core component of the modern data stack.

To be of high quality, data must be consistent and unambiguous. You can measure data quality through dimensions…

Data scientists and data engineers are responsible for authoring data pipelines and workflows. Historically individuals wrote cron jobs to orchestrate data but today there are data orchestration frameworks that allow them to programmatically author, schedule, and monitor data pipelines. Over the past few years, we have seen the emergence of numerous data orchestration frameworks and believe it is a core component of the modern data stack.

Until recently, data teams used cron to schedule data jobs. However, as data teams began writing more cron jobs the growing number and complexity became hard to manage. In particular, managing dependencies between…