├── BuildingInspections ├── README.md └── data │ ├── Building_Violations_50000_features_and_labels.csv │ ├── Building_Violations_sample_50000.csv │ └── Buildings_Violations_sample_50000_clean.csv ├── ChicagoCityOfLearning ├── .DS_Store ├── README.md ├── data │ └── chicago_acs.geojson ├── data_clean │ ├── schedule_program_export.csv │ ├── schedule_program_export.xls │ ├── schedule_program_export_cleaner.csv │ └── schedule_program_export_cleaner.xls ├── learnathon │ └── distances.sql └── open_refine_export │ └── schedule_program_export.openrefine.tar.gz ├── README.md └── RetrofitChicagoTargeting └── README.md /BuildingInspections/README.md: -------------------------------------------------------------------------------- 1 | ![Project Inspector Gadget](http://www.orlando-home-inspection-services.com/images/inspector-gadget.jpg) 2 | 3 | #Project Inspector Gadget: Targeted Building Inspections for the City of Chicago 4 | 5 | ###Project Description 6 | 7 | The City of Chicago's [Department of Buildings](http://www.cityofchicago.org/city/en/depts/bldgs.html/) supports the safety and quality of life for the residents and visitors of the City of Chicago through enforcement of the [Chicago Building Code](https://www.cityofchicago.org/city/en/depts/bldgs/provdrs/inspect/svcs/chicago_buildingcodeonline.html). The Department of Buildings issues building permits and performs regular inspections of buildings throughout the city as part of this mandate. The permitting and [inspection](http://www.cityofchicago.org/city/en/depts/bldgs/provdrs/inspect/svcs/request_an_inspection.html) process promotes high quality design standards as well as the conservation, rehabilitation and reuse of the City's existing buildings. There are lots of buildings to inspect. Given its [prior success in building models for predictive inspection with other City departments](https://github.com/Chicago/food-inspections-evaluation) the Department of Innovation and Technology would like to work with the Buildings Department to develop a predictive model that helps building inspectors focus their inspections on the the buildings that are most likely to be involution of building regulations. Our goal is to build a predictive model for the City of Chicago that tells inspectors which buildings they should inspect next week. 8 | 9 | ####Objectives 10 | 11 | Help the City of Chicago understand and improve the building inspection process by 12 | 13 | - Identifying patterns in building inspections and violations 14 | - Building a complete data science pipeline that produces an operationally useful list of which buildings the City’s building department should inspect 15 | 16 | ####Data Sources 17 | 18 | - [City of Chicago Building Permits](https://data.cityofchicago.org/Buildings/Building-Permits/ydr8-5enu) (Source: City of Chicago Data Portal) 19 | - [City of Chicago Building Violations](https://data.cityofchicago.org/Buildings/Building-Violations/22u3-xenr) (Source: City of Chicago Data Portal) 20 | - [Demographic information by Census Tract](https://raw.githubusercontent.com/dssg/data-challenges/master/ChicagoCityOfLearning/data/chicago_acs.geojson) (Source: American FactFinder) 21 | - [Census tract shapefiles](https://www.census.gov/geo/maps-data/data/tiger-line.html) 22 | 23 | ####Tasks 24 | - The complete task and resource list is found in [this hackpad](https://hackpad.com/DSSG-2015-Bootcamp-Tasks-and-Resources-oSt57twcmfY) 25 | 26 | ####Tools Used 27 | - git/Github 28 | - commandline tools / csvkit 29 | - PostgreSQL/PostGIS 30 | - Python: Pandas 31 | - Python: sklearn 32 | - Python: matplotlib 33 | - R: plyr 34 | - R: ggplot2 35 | - R: glm 36 | - Tableau 37 | 38 | 39 | ####Concepts 40 | - Working with data portals (Basic) 41 | - Connecting to an open data portal API (Basic) 42 | - Command Line (Basic) 43 | - Data cleaning (Basic) 44 | - Databases (Intermediate) 45 | - Data Exploration in Python (Basic) 46 | - Data Exploration in R (Basic) 47 | - Data Visualization (Basic) 48 | - Model Building in Python (Basic) 49 | - Model Building in R (Basic) 50 | - Machine Learning: Prediction (Basic) 51 | - Github (Basic) 52 | 53 | -------------------------------------------------------------------------------- /ChicagoCityOfLearning/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dssg/data-challenges/59013da39838f845f768bd2581ef21f664b29bc5/ChicagoCityOfLearning/.DS_Store -------------------------------------------------------------------------------- /ChicagoCityOfLearning/README.md: -------------------------------------------------------------------------------- 1 | #DSSG Data Challenge: Improving Summer Learning Opportunities for Chicago City of Learning 2 | 3 | ###Project Description 4 | 5 | The City of Chicago last year piloted a major initiative called Chicago Summer of Learning aimed at helping school children and their parents better navigate summering learning opportunities, and receive credit for their summer achievements through Mozilla's Open Badging platform. 6 | 7 | For the first time, program administrators have a comprehensive database of all summer learning opportunities in the city. They are interested in knowing whether the current distribution of activities provides equal access to children throughout the city. 8 | 9 | ####Objectives 10 | 11 | Help the City of Chicago understand and improve access to summer learning opportunities by: 12 | 13 | - Describing current distibution of summer learning opportunities and their overall accesability 14 | - Prescribing potential areas for changing grant incentives to focus on certain types of programs to increase equality of access across the city 15 | 16 | ####Data Sources 17 | 18 | - [Location of Summer Learning Programs](http://files.figshare.com/1515450/schedule_program_export.csv) (Source: Digital Media and Learning Lab at Depaul) 19 | - [Location of Schools]( https://data.cityofchicago.org/Education/CPS-Schools-2013-2014-Academic-Year/c7jj-qjvh) (Source: Chicago's Data Portal) 20 | - [Demographic information by Census Tract](https://raw.githubusercontent.com/dssg/data-challenges/master/ChicagoCityOfLearning/data/chicago_acs.geojson) (Source: American FactFinder) 21 | - [Location of Summer Learning Programs - Cleaned Version](https://github.com/dssg/data-challenges/raw/master/ChicagoCityOfLearning/data_clean/schedule_program_export.xls) (Source: Digital Media and Learning Lab at Depaul) 22 | 23 | ####Tasks 24 | 25 | - Extraction: Download Summer Program Data from Figshare 26 | - Cleaning: Examine and clean the data of duplicates and exporting errors using Google Refine (data cleaning) 27 | - Documentation: Upload cleaned data to Clean Data (or dat) (https://docs.google.com/spreadsheets/d/1sKuGhXyAQk2iQHqqA8ZV5GWqOOPnPivRBJjmLWMIn1o/edit?usp=sharing) 28 | - Visualization: Upload the dataset to fusion tables ( https://www.google.com/fusiontables/DataSource?docid=1beH1546qrZc_J6fd2ClC4uph7HzZUH66kvZ_wiqL ) 29 | - Exploration: Load cleaned data into R or Python and generate summary statistics 30 | - Extraction: Use python or R to connect to the City's Open Data Portal & download locations of all public schools in the city 31 | - Storage: Load both datasets into a PostgreSQL database 32 | - Analysis: Calculate the number of programs within a 1 mile radius of each school. 33 | - AnalysisCalculate the average distance that a student attending a particular school would have to travel 34 | - Extraction: Download census tract-level data 35 | - Analysis: Explore the correlation between local demographics and program accessability 36 | - Visualization: Create a visualization that shows relationship between program counts and average distance traveled, area demographics 37 | - Proscription: Generate a list of areas where the addition of a program could increase overall equality of opportunity 38 | - Proscription: Develop alternative allocation of summer programs that would be more equitable and minimize travel time for students from all schools 39 | 40 | ####Tools Used 41 | - Open Refine 42 | - Google Fusion Tables 43 | - Python: Pandas 44 | - Python: statsmodels 45 | - Python: matplotlib 46 | - PostgreSQL 47 | - PostGIS 48 | 49 | 50 | ####Concepts 51 | - Connecting to an open data portal API (Basic) 52 | - Data cleaning (Basic) 53 | - Databases (Intermediate) 54 | - Geospatial analysis (Basic) 55 | - Simulation/Optimization (Advanced) 56 | 57 | 58 | -------------------------------------------------------------------------------- /ChicagoCityOfLearning/data_clean/schedule_program_export.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dssg/data-challenges/59013da39838f845f768bd2581ef21f664b29bc5/ChicagoCityOfLearning/data_clean/schedule_program_export.csv -------------------------------------------------------------------------------- /ChicagoCityOfLearning/data_clean/schedule_program_export.xls: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dssg/data-challenges/59013da39838f845f768bd2581ef21f664b29bc5/ChicagoCityOfLearning/data_clean/schedule_program_export.xls -------------------------------------------------------------------------------- /ChicagoCityOfLearning/data_clean/schedule_program_export_cleaner.csv: -------------------------------------------------------------------------------- 1 | name,org_name,description,address,city,state,zipcode,latitude,longitude,min_age,max_age,category_id,category name,category description,Column,Column2,Column3,Column4,Column5,Column6,Column7,Column8,Column9,Column10,Column11,Column12,Column13,Column14,Column15,Column16,Column17 2 | CPS Virtual Learning ,Chicago Public Schools,"Online courses for CPS students (rising 7th and 8th graders) to take non-credit bearing online courses to enhance math, writing and reading skills over the summer. Courses are taken online anytime anywhere for students applicants who apply or those nominated by their counselor, parent or guardian.",,,IL,,NULL,NULL,12,16,11,Numbers,Using math to explore the world,,,,,,,,,,,,,,,,, 3 | "CPS Virtual Learning \Be Prepared\"" Program""",Chicago Public Schools,"Online courses for CPS students (rising 7th and 8th graders) to take non-credit bearing online courses to enhance math, writing and reading skills over the summer. Courses are taken online anytime anywhere for students applicants who apply or those nominated by their counselor, parent or guardian.",,,IL,,NULL,NULL,12,16,11,Numbers,Using math to explore the world,,,,,,,,,,,,,,,,, 4 | "CPS Virtual Learning \Be Prepared\"" Program""",Chicago Public Schools,"Online courses for CPS students (rising 7th and 8th graders) to take non-credit bearing online courses to enhance math, writing and reading skills over the summer. Courses are taken online anytime anywhere for students applicants who apply or those nominated by their counselor, parent or guardian.",,,IL,,NULL,NULL,12,16,9,Storytelling,Crafting and understanding fictional + historical stories,,,,,,,,,,,,,,,,, 5 | "CPS Virtual Learning \Be Prepared\"" Program""",Chicago Public Schools,"Online courses for CPS students (rising 7th and 8th graders) to take non-credit bearing online courses to enhance math, writing and reading skills over the summer. Courses are taken online anytime anywhere for students applicants who apply or those nominated by their counselor, parent or guardian.",,,IL,,NULL,NULL,12,16,11,Numbers,Using math to explore the world,,,,,,,,,,,,,,,,, 6 | Hip-Hop Yoga,Mindful Practices,"A brilliant blending of two seemingly disparate cultures, Hip-Hop Yoga, fuses hip-hop dance and yoga into a fun, creative performance piece. This revolutionary program offers participants the mind-body centering and relaxation elements of yoga combined with hip-hop's high energy, athletic choreography. Participants work collaboratively throughout the workshop to develop a performance piece.",,,IL,,40.6331,-89.3985,5,24,8,Performance,"Participating in live music, theatre, and dance performances",,,,,,,,,,,,,,,,, 7 | Hip-Hop Yoga,Mindful Practices,"A brilliant blending of two seemingly disparate cultures, Hip-Hop Yoga, fuses hip-hop dance and yoga into a fun, creative performance piece. This revolutionary program offers participants the mind-body centering and relaxation elements of yoga combined with hip-hop's high energy, athletic choreography. Participants work collaboratively throughout the workshop to develop a performance piece.",,,IL,,40.6331,-89.3985,5,24,5,Sports + Wellness,"Playing sports, helping the environment, and other healthy activities",,,,,,,,,,,,,,,,, 8 | Hip-Hop Yoga,Mindful Practices,"A brilliant blending of two seemingly disparate cultures, Hip-Hop Yoga, fuses hip-hop dance and yoga into a fun, creative performance piece. This revolutionary program offers participants the mind-body centering and relaxation elements of yoga combined with hip-hop's high energy, athletic choreography. Participants work collaboratively throughout the workshop to develop a performance piece.",,,IL,,40.6331,-89.3985,5,24,8,Performance,"Participating in live music, theatre, and dance performances",,,,,,,,,,,,,,,,, 9 | Hip-Hop Yoga,Mindful Practices,"A brilliant blending of two seemingly disparate cultures, Hip-Hop Yoga, fuses hip-hop dance and yoga into a fun, creative performance piece. This revolutionary program offers participants the mind-body centering and relaxation elements of yoga combined with hip-hop's high energy, athletic choreography. Participants work collaboratively throughout the workshop to develop a performance piece.",,,IL,,40.6331,-89.3985,5,24,5,Sports + Wellness,"Playing sports, helping the environment, and other healthy activities",,,,,,,,,,,,,,,,, 10 | -------------------------------------------------------------------------------- /ChicagoCityOfLearning/data_clean/schedule_program_export_cleaner.xls: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dssg/data-challenges/59013da39838f845f768bd2581ef21f664b29bc5/ChicagoCityOfLearning/data_clean/schedule_program_export_cleaner.xls -------------------------------------------------------------------------------- /ChicagoCityOfLearning/learnathon/distances.sql: -------------------------------------------------------------------------------- 1 | 2 | SELECT sl.school, min(ST_Distance(sl.the_geom::geography, asp.the_geom::geography)) 3 | FROM schoollocations2012_13 sl 4 | CROSS JOIN schedule_program_export asp 5 | GROUP BY sl.school 6 | ORDER BY 2 DESC 7 | -------------------------------------------------------------------------------- /ChicagoCityOfLearning/open_refine_export/schedule_program_export.openrefine.tar.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dssg/data-challenges/59013da39838f845f768bd2581ef21f664b29bc5/ChicagoCityOfLearning/open_refine_export/schedule_program_export.openrefine.tar.gz -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | DSSG Data Challenges 2 | =============== 3 | 4 | Data Challenges are real-world data challenges faced by organizations that can be solved using real data. The primary purpose of a data challenge is to provide a motivating example for the project-based learning environment at DSSG. The secondary outcome is that solutions developed while working on the challenge can be implemented by local partners to improve their programs. 5 | -------------------------------------------------------------------------------- /RetrofitChicagoTargeting/README.md: -------------------------------------------------------------------------------- 1 | #DSSG Data Challenge: Improving the Effectiveness of Publicly-funded Energy Efficiency Programs by Identifying Target Areas and Buildings with the Greatest Need 2 | 3 | ###Project Description 4 | 5 | The City of Chicago is engaged in a major initiative called Retrofit Chicago aimed at reducing energy waste and lowing energy bills for Chicago Residents. 6 | 7 | Low income residents are disproportionately affected by energy waste. Low income households in Chicago pay up to a quarter of their total takehome earnings in energy costs. Helping them save on energy means they have more to spend on food, healthcare, and education for their kids. 8 | 9 | ####Objectives 10 | 11 | Help the City of Chicago's Retrofit Chicago initiative understand and improve the effectiveness of energy efficiency retrofit programs by: 12 | 13 | - Describing the energy efficiency potential 14 | - Predicting homes and areas are most likely to have the greatest energy efficiency potential and 15 | - Prescribing a list of target homes that weights energy efficiency potential and social impact (savings on bills as a percentage of income) 16 | 17 | 18 | ####Data 19 | 20 | * [Energy Intensity of Buildings by Block](https://data.cityofchicago.org/Environment-Sustainable-Development/Energy-Usage-2010/8yq3-m6wp) (Source: City of Chicago Data Portal) 21 | * Location of previous retrofits in the city (Source: Delta institute) 22 | * [Building footprints](https://data.cityofchicago.org/Buildings/Building-Footprints/qv97-3bvb) (Source: City of Chicago Data Portal) 23 | * [County Assessor's Data]() (Source: Depaul Institute of Housing Studies) 24 | * [Census Tract Demographic](http://censusreporter.org/) Information (Source: Census Reporter) 25 | 26 | ####Tasks 27 | 28 | --------------------------------------------------------------------------------