Code for Philly Data Pipeline
Stage
README
We're building a data pipeline + dashboard for Code for Philly data!
The pipeline has two major goals:
- make it easier for community members to track active projects.
- serve as a template for other projects.
Background
Code for Philly (CfP) is a place where volunteers and non-profit organizations collaborate on projects. Since its founding in 2012, it’s seen a lot of activity:
- community slack with 3,000+ members.
- codeforphilly.org listing 246 projects.
- github.com/CodeForPhilly with 113 repos.
However, as CfP has grown, it’s become increasingly difficult for volunteers to find active projects to join, and for CfP leadership to keep track of progress across projects.
Roles Needed
- Engineers to help build and explore pipeline tools (e.g. with airflow).
- Data scientists / analysts to help with the dashboard and reporting.
- Product managers / UX researchers to ensure it's useful to CfP members and projects.
Don't fit the above? Join the slack channel, and we'll find something in the project for you!
Meeting notes
We meet once a week. Below are some notes from past meetings.
| week | discussion | links |
|---|---|---|
| 12 Jan 2021 | Kicked off the project! | 10 minute presentation on project (Michael @machow) |
| 19 Jan 2021 | Darrel Herbst (@dherbst) reviewed his deep dive into the pipelining tools meltano and singer taps | He forked github repos for us to tap github and tap slack data. |
| 3 Feb 2021 | Introduced cfpr, which allows analysts to connect to our data in RStudio | cfpr github repo |
| 23 Feb 2021 | Sophie Finn (@sophfinn) ran us through a design exercise based on the 5E's experience model | TODO: screencast summarizing result or link to a document |
