SFBigAnalytics_20190410: Apache Airflow At Lyft HD

Apache Airflow At Lyft Lyft has been one of the first companies to adopt Airflow in production. Today Airflow powers many Lyft use cases: from powering executive dashboards to metrics aggregation, to derived data generation, to machine learning feature computation, etc. In this talk, we will first cover how we operate Airflow at Lyft in production, then we will talk about the improvement we have done for Airflow to boost internal ETL development productivity. Lastly, we will talk about some of our open source contributions which could benefit the whole community. Speaker: Tao Feng, Tao Feng is a software engineer at Lyft data platform team working on various data products. Tao is also a committer and PMC on Apache Airflow. Previously, Tao worked at Linkedin and oracle on data infrastructure, tooling and performance.