├── .gitattributes ├── .gitignore ├── .travis.yml ├── AvianDietDatabase.txt ├── AvianDietDatabase_Cornell2023.txt ├── AvianDietDatabase_Gonzalez1991.txt ├── AvianDietDatabase_Gonzalez1991_clean_clean.txt ├── AvianDietDatabase_nonNorthAmerica.txt ├── AvianDietDatabase_template.txt ├── DietDatabase_YoungandBlomme1975.txt ├── DietDatabase_YoungandBlomme1975_badnames.txt ├── DietDatabase_YoungandBlomme1975_clean.txt ├── Fischer_1958_Cleaned.txt ├── Gonzalez1991_cleaned.txt ├── Kent_2022_Clean.txt ├── LICENSE ├── NA_avian_diet_refs.txt ├── README.md ├── archive ├── AvianDietDatabase_old.txt ├── Olsen2015-AppendixS1.xls ├── Olsen2015-AppendixS2.xls ├── diet_compendia.txt ├── eBird_Taxonomy_v1.55.txt ├── felder.tsv └── testDatabase.txt ├── birdtaxonomy ├── ABA_Checklist-8.0.6a.csv ├── Clements_Taxonomy_v2018_NorthAmerica.csv ├── NA_specieslist.csv ├── aba_to_ebird_taxonomy_conversion.csv ├── birdies.csv ├── creating_NA_specieslist.r ├── eBird-Clements-v2021-integrated-checklist-August-2021.csv ├── eBird_Taxonomy_v2016.csv ├── eBird_Taxonomy_v2016_NorthAmerica.csv ├── eBird_Taxonomy_v2018.csv ├── eBird_Taxonomy_v2019.csv ├── eBird_Taxonomy_v2021.csv ├── ebird_taxonomy_v2021_subset.csv └── orders.csv ├── cleaning ├── AvianDietDatabase_unstandardized_testfile.txt ├── Database_notes_and_errors.md ├── Olsen_db_cleaning_history.r ├── beal_cleaning.R ├── cleaning_checksum_flagged_records.txt ├── cleaning_overview.md ├── compare_previous_db.R ├── database_error_checking.R ├── db_cleaning_history.r ├── fraction_sum_check_cleaning.txt ├── martin1961_format_conversion.r ├── name_changes.txt ├── name_cleaning_instructions.md ├── name_cleaning_test_data.txt ├── prey_parts.txt ├── prey_stages.txt ├── problem_names.txt ├── qa_qc_test_db.txt ├── taxonUnmatched.tsv ├── test_namereplace_db.txt ├── test_namereplace_db_badnames.txt ├── test_namereplace_db_clean.txt └── test_probnames.txt ├── contributors.md ├── data_priorities.md ├── dietdatabase.Rproj ├── globi.json ├── in_progress ├── AvianDietDatabase_Bradstreet_1980.txt ├── AvianDietDatabase_Schneider_and_Pence_1992.txt ├── OTU_dataset.csv ├── OTU_taxonomy.csv ├── Stillman_et_al_2022_OTU_count.csv ├── Stillman_et_al_2022_raw_woodpecker_data.csv ├── fakeclean.txt └── reshape_Stillman_et_al_raw_data.r ├── instructions ├── cloning_repo_with_RStudio.md ├── git_dietdatabase_help.md ├── instructions.md ├── ssh_instructions.md └── training_instructions.md ├── preyCommonNames.csv ├── scripts ├── bird_species_list.r ├── database_summary_functions.R ├── name_cleaning_test_data.txt ├── occurrence_summary_test_data.txt └── prey_common_names.r └── training ├── AvianDietDatabase_AR.txt.txt ├── AvianDietDatabase_AR2.txt ├── AvianDietDatabase_RLC.txt ├── AvianDietDatabase_training_newKEY.txt ├── AvianDietDatabase_training_newKEY_CB.txt ├── training_dietdatabase.txt ├── training_dietdatabase_AD.txt ├── training_dietdatabase_CS.txt ├── training_dietdatabase_EK.txt ├── training_dietdatabase_HL.txt ├── training_dietdatabase_JC.txt ├── training_dietdatabase_JSW.txt ├── training_dietdatabase_JSW_badnames.txt ├── training_dietdatabase_JSW_clean.txt ├── training_dietdatabase_KEY.txt ├── training_dietdatabase_LP.txt ├── training_dietdatabase_MAS.txt ├── training_dietdatabase_RLQ.txt ├── training_dietdatabase_TH.txt ├── training_dietdatabase_YP.txt ├── training_dietdatabase_wl.txt └── training_summary_notes.md /.gitattributes: -------------------------------------------------------------------------------- 1 | AvianDietDatabase.txt -text 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .Rhistory 2 | .Rproj.user 3 | .RData 4 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | install: 2 | - wget "https://raw.githubusercontent.com/globalbioticinteractions/globinizer/master/check-dataset.sh" -O check-dataset.sh 3 | - chmod +x check-dataset.sh 4 | 5 | script: ./check-dataset.sh ${TRAVIS_REPO_SLUG} 6 | 7 | notifications: 8 | email: 9 | - jhpoelen+avian@gmail.com 10 | - hurlbert@bio.unc.edu 11 | -------------------------------------------------------------------------------- /AvianDietDatabase_Cornell2023.txt: -------------------------------------------------------------------------------- 1 | Common_Name Scientific_Name Subspecies Family Taxonomy Longitude_dd Latitude_dd Altitude_min_m Altitude_mean_m Altitude_max_m Location_Region Location_Specific Habitat_type Observation_Month_Begin Observation_Year_Begin Observation_Month_End Observation_Year_End Observation_Season Analysis_Number Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Inclusive_Prey_Taxon Prey_Name_ITIS_ID Prey_Name_Status Prey_Stage Prey_Part Prey_Common_Name Fraction_Diet Diet_Type Item_Sample_Size Bird_Sample_Size Sites Study_Type Notes Entered_By Source doi sex age_class within_study_data_source table_fig_number title lastname_author source_year journal 2 | American Kestrals Falco sparverius Falconidae eBird Clements checklist v2021 NA NA NA Pennsylvania Southeastern Pennsylvania agriculture 5 2019 6 2019 Summer 157 Animalia Chordata Aves Passeriformes 1 behavioral observation SC "Cornell, A., Fowler, M. A., Zimmerman, C., Khaku, Z., & Therrien, J. (2023). The role of food quantity and prey type in nestling development of American kestrels. Journal of Raptor Research, 57(2). https://doi.org/10.3356/jrr-22-10" https://doi.org/10.3356/JRR-22-10 Chick The Role of Food Quantity and Prey Type in Nestling Development of American Kestrels Cornell 2023 Journal of Raptor Research 3 | 157 Animalia Chordata Mammalia 1 4 | 157 Animalia Arthropoda 1 5 | 314 Animalia Chordata Mammalia 6 | 630 Animalia Chordata Aves Passeriformes 7 | 630 Animalia Arthropoda 8 | 633 Animalia Chordata Aves Passeriformes 9 | 633 Animalia Arthropoda 10 | 633 Animialia Annelida 11 | 636 Animalia Chordata Aves Passeriformes 12 | 636 Animalia Arthropoda 13 | 636 Animialia Annelida 14 | 636 Animalia Chordata Mammalia 15 | 639 Animalia Chordata Aves Passeriformes 16 | 639 Animalia Arthropoda 17 | 639 Animialia Annelida 18 | 639 Animalia Chordata Mammalia 19 | 661 Animalia Chordata Aves Passeriformes 20 | 661 Animalia Arthropoda 21 | 661 Animialia Annelida 22 | 661 Animalia Chordata Mammalia 23 | 668 Animalia Chordata Aves Passeriformes 24 | 668 Animalia Arthropoda 25 | 668 Animalia Chordata Mammalia 26 | 668 27 | 704 Animalia Arthropoda 28 | 704 Animalia Chordata Mammalia 29 | 704 30 | 901 Animalia Arthropoda 31 | 901 Animialia Annelida 32 | 901 Animalia Chordata Mammalia 33 | -------------------------------------------------------------------------------- /AvianDietDatabase_nonNorthAmerica.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/AvianDietDatabase_nonNorthAmerica.txt -------------------------------------------------------------------------------- /AvianDietDatabase_template.txt: -------------------------------------------------------------------------------- 1 | Common_Name Scientific_Name Subspecies Family Taxonomy Longitude_dd Latitude_dd Altitude_min_m Altitude_mean_m Altitude_max_m Location_Region Location_Specific Habitat_type Observation_Month_Begin Observation_Year_Begin Observation_Month_End Observation_Year_End Observation_Season Analysis_Number Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Inclusive_Prey_Taxon Prey_Name_ITIS_ID Prey_Name_Status Prey_Stage Prey_Part Prey_Common_Name Fraction_Diet Diet_Type Item_Sample_Size Bird_Sample_Size Sites Study_Type Notes Entered_By Source 2 | -------------------------------------------------------------------------------- /DietDatabase_YoungandBlomme1975_badnames.txt: -------------------------------------------------------------------------------- 1 | "level" "name" "condition" 2 | "Scientific_Name" "Rana pipiens" "unmatched" 3 | "Scientific_Name" "Eutamias minimus" "unmatched" 4 | "Scientific_Name" "Capella gallinago" "unmatched" 5 | "Scientific_Name" "Circotettix verruculatus" "unmatched" 6 | "Family" "Cicindelidae" "unmatched" 7 | "Order" "Gastropoda" "wrong rank; too low" 8 | -------------------------------------------------------------------------------- /Fischer_1958_Cleaned.txt: -------------------------------------------------------------------------------- 1 | "Common_Name" "Scientific_Name" "Subspecies" "Family" "Taxonomy" "Longitude_dd" "Latitude_dd" "Altitude_min_m" "Altitude_mean_m" "Altitude_max_m" "Location_Region" "Location_Specific" "Habitat_type" "Observation_Month_Begin" "Observation_Year_Begin" "Observation_Month_End" "Observation_Year_End" "Observation_Season" "Analysis_Number" "Prey_Kingdom" "Prey_Phylum" "Prey_Class" "Prey_Order" "Prey_Suborder" "Prey_Family" "Prey_Genus" "Prey_Scientific_Name" "Inclusive_Prey_Taxon" "Prey_Name_ITIS_ID" "Prey_Name_Status" "Prey_Stage" "Prey_Part" "Prey_Common_Name" "Fraction_Diet" "Diet_Type" "Item_Sample_Size" "Bird_Sample_Size" "Sites" "Study_Type" "Notes" "Entered_By" "Source" "Doi" "Sex" "Age_Class" "Within_Study_Data_Source" "Table_Fig_Number" "Title" "Lastname_author" "Source_year" "Journal" 2 | "Chimney Swift" "Chaetura pelagica" NA "Apodidae" "eBird Clements checklist v2021" NA NA NA 487.68 NA "New York" "Beaver Kill in Rockland, New York" "deciduous forest;agriculture" NA 1950 NA 1952 NA 0.95 "Animalia" "Arthropoda" "Insecta" "" "" "" "" "" "yes" NA NA NA NA NA 1 "" 1000 NA 7 "behavioral observation; pellet contents" "specifically noted Diptera, Hymenoptera, Plecoptera, Coleoptera, Hemiptera, Trichoptera, and Siphonaptera." "AM" "Fischer, R. B. (1958). The Breeding Biology of the Chimney Swift. New York State Museum and Science Service Bulletin Number 368." NA NA NA "In the text" NA "The Breeding Biology of the Chimney Swift" "Fischer" 1958 "New York State Museum and Science Service Bulletin" 3 | "Chimney Swift" "Chaetura pelagica" NA "Apodidae" "eBird Clements checklist v2021" NA NA NA 487.68 NA "New York" "Beaver Kill in Rockland, New York" "deciduous forest;agriculture" NA 1950 NA 1952 NA 0.01 "Animalia" "Arthropoda" "Arachnida" "Araneae" "" "" "" "" "yes" NA NA NA NA NA 1 "" 1000 NA 7 "behavioral observation; pellet contents" "" "AM" "Fischer, R. B. (1958). The Breeding Biology of the Chimney Swift. New York State Museum and Science Service Bulletin Number 368." NA NA NA "In the text" NA "The Breeding Biology of the Chimney Swift" "Fischer" 1958 "New York State Museum and Science Service Bulletin" 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Creative Commons Legal Code 2 | 3 | CC0 1.0 Universal 4 | 5 | CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE 6 | LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN 7 | ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS 8 | INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES 9 | REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS 10 | PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM 11 | THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED 12 | HEREUNDER. 13 | 14 | Statement of Purpose 15 | 16 | The laws of most jurisdictions throughout the world automatically confer 17 | exclusive Copyright and Related Rights (defined below) upon the creator 18 | and subsequent owner(s) (each and all, an "owner") of an original work of 19 | authorship and/or a database (each, a "Work"). 20 | 21 | Certain owners wish to permanently relinquish those rights to a Work for 22 | the purpose of contributing to a commons of creative, cultural and 23 | scientific works ("Commons") that the public can reliably and without fear 24 | of later claims of infringement build upon, modify, incorporate in other 25 | works, reuse and redistribute as freely as possible in any form whatsoever 26 | and for any purposes, including without limitation commercial purposes. 27 | These owners may contribute to the Commons to promote the ideal of a free 28 | culture and the further production of creative, cultural and scientific 29 | works, or to gain reputation or greater distribution for their Work in 30 | part through the use and efforts of others. 31 | 32 | For these and/or other purposes and motivations, and without any 33 | expectation of additional consideration or compensation, the person 34 | associating CC0 with a Work (the "Affirmer"), to the extent that he or she 35 | is an owner of Copyright and Related Rights in the Work, voluntarily 36 | elects to apply CC0 to the Work and publicly distribute the Work under its 37 | terms, with knowledge of his or her Copyright and Related Rights in the 38 | Work and the meaning and intended legal effect of CC0 on those rights. 39 | 40 | 1. Copyright and Related Rights. A Work made available under CC0 may be 41 | protected by copyright and related or neighboring rights ("Copyright and 42 | Related Rights"). Copyright and Related Rights include, but are not 43 | limited to, the following: 44 | 45 | i. the right to reproduce, adapt, distribute, perform, display, 46 | communicate, and translate a Work; 47 | ii. moral rights retained by the original author(s) and/or performer(s); 48 | iii. publicity and privacy rights pertaining to a person's image or 49 | likeness depicted in a Work; 50 | iv. rights protecting against unfair competition in regards to a Work, 51 | subject to the limitations in paragraph 4(a), below; 52 | v. rights protecting the extraction, dissemination, use and reuse of data 53 | in a Work; 54 | vi. database rights (such as those arising under Directive 96/9/EC of the 55 | European Parliament and of the Council of 11 March 1996 on the legal 56 | protection of databases, and under any national implementation 57 | thereof, including any amended or successor version of such 58 | directive); and 59 | vii. other similar, equivalent or corresponding rights throughout the 60 | world based on applicable law or treaty, and any national 61 | implementations thereof. 62 | 63 | 2. Waiver. To the greatest extent permitted by, but not in contravention 64 | of, applicable law, Affirmer hereby overtly, fully, permanently, 65 | irrevocably and unconditionally waives, abandons, and surrenders all of 66 | Affirmer's Copyright and Related Rights and associated claims and causes 67 | of action, whether now known or unknown (including existing as well as 68 | future claims and causes of action), in the Work (i) in all territories 69 | worldwide, (ii) for the maximum duration provided by applicable law or 70 | treaty (including future time extensions), (iii) in any current or future 71 | medium and for any number of copies, and (iv) for any purpose whatsoever, 72 | including without limitation commercial, advertising or promotional 73 | purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each 74 | member of the public at large and to the detriment of Affirmer's heirs and 75 | successors, fully intending that such Waiver shall not be subject to 76 | revocation, rescission, cancellation, termination, or any other legal or 77 | equitable action to disrupt the quiet enjoyment of the Work by the public 78 | as contemplated by Affirmer's express Statement of Purpose. 79 | 80 | 3. Public License Fallback. Should any part of the Waiver for any reason 81 | be judged legally invalid or ineffective under applicable law, then the 82 | Waiver shall be preserved to the maximum extent permitted taking into 83 | account Affirmer's express Statement of Purpose. In addition, to the 84 | extent the Waiver is so judged Affirmer hereby grants to each affected 85 | person a royalty-free, non transferable, non sublicensable, non exclusive, 86 | irrevocable and unconditional license to exercise Affirmer's Copyright and 87 | Related Rights in the Work (i) in all territories worldwide, (ii) for the 88 | maximum duration provided by applicable law or treaty (including future 89 | time extensions), (iii) in any current or future medium and for any number 90 | of copies, and (iv) for any purpose whatsoever, including without 91 | limitation commercial, advertising or promotional purposes (the 92 | "License"). The License shall be deemed effective as of the date CC0 was 93 | applied by Affirmer to the Work. Should any part of the License for any 94 | reason be judged legally invalid or ineffective under applicable law, such 95 | partial invalidity or ineffectiveness shall not invalidate the remainder 96 | of the License, and in such case Affirmer hereby affirms that he or she 97 | will not (i) exercise any of his or her remaining Copyright and Related 98 | Rights in the Work or (ii) assert any associated claims and causes of 99 | action with respect to the Work, in either case contrary to Affirmer's 100 | express Statement of Purpose. 101 | 102 | 4. Limitations and Disclaimers. 103 | 104 | a. No trademark or patent rights held by Affirmer are waived, abandoned, 105 | surrendered, licensed or otherwise affected by this document. 106 | b. Affirmer offers the Work as-is and makes no representations or 107 | warranties of any kind concerning the Work, express, implied, 108 | statutory or otherwise, including without limitation warranties of 109 | title, merchantability, fitness for a particular purpose, non 110 | infringement, or the absence of latent or other defects, accuracy, or 111 | the present or absence of errors, whether or not discoverable, all to 112 | the greatest extent permissible under applicable law. 113 | c. Affirmer disclaims responsibility for clearing rights of other persons 114 | that may apply to the Work or any use thereof, including without 115 | limitation any person's Copyright and Related Rights in the Work. 116 | Further, Affirmer disclaims responsibility for obtaining any necessary 117 | consents, permissions or other rights required for any use of the 118 | Work. 119 | d. Affirmer understands and acknowledges that Creative Commons is not a 120 | party to this document and has no duty or obligation with respect to 121 | this CC0 or use of the Work. 122 | -------------------------------------------------------------------------------- /NA_avian_diet_refs.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/NA_avian_diet_refs.txt -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Avian Diet Database 2 | =================== 3 | 4 | [![DOI](https://zenodo.org/badge/22651884.svg)](https://zenodo.org/badge/latestdoi/22651884) 5 | [![License: CC0-1.0](https://licensebuttons.net/l/zero/1.0/80x15.png)](http://creativecommons.org/publicdomain/zero/1.0/) 6 | 7 | This database is a quantitative summary of the published literature on the diets of bird species. 8 | 9 | ## Interactive Website 10 | 11 | This website ([https://aviandiet.unc.edu](https://aviandiet.unc.edu)) allows users to explore the most up-to-date database information, and to generate basic summaries by diet or prey item. 12 | 13 | ## R Package 14 | A developmental **R package** is available [here](https://github.com/ahhurlbert/aviandietdb) for exploring, summarizing, and filtering the database in the R environment. 15 | 16 | Note that the `dietdb` database object loaded with this package may be slightly out of date relative to the `AvianDietDatabase.txt` file in this repository. 17 | 18 | ## Data Entry 19 | 20 | Instructions for entering and cleaning new diet data information can be found [here](instructions/instructions.md). 21 | 22 | Did you find an error in the database? Do you know of a published study with quantitative diet information that is not included in our database? 23 | 24 | Let us know! Post an issue by clicking ‘Issues’ near the top left (requires a Github account), or send an email to aviandietdb@gmail.com. 25 | 26 | ## Citation 27 | 28 | Please cite this dataset as: 29 | 30 | Hurlbert, A. H., Olsen, A. M., Sawyer, M. M., and Winner, P. M. 2021. Avian Diet Database. Scientific Data 8: 260. https://www.nature.com/articles/s41597-021-01049-9 31 | 32 | 33 | 34 | ## Acknowledgments 35 | 36 | Many individuals contributed to this compilation of diet records. Specifically we would like to thank M. Auriemma, J. Barnes, M. Blawas, C. Bridges, J. Cook, R. Cooper, A. Duprey, J. Fox, G. Glass, T. Hennessey, J. Knowlton, E. Kremer, W. Larsen, E. McGeady, T. McKinnon, G. Ni, L. Peeler, R. Quindlen, A. Richards, D. Ryu, M. Sandoval, C. Seyfried, S. Swinea, and J. Willard, A. Zachman. M. Pan, T. Randby, and T. Le developed the web interface for exploring the database. AHH also acknowledges support from NSF DEB-1354563 and NSF EF-1702708. 37 | 38 | ## Indexing 39 | [![GloBI Index Status](https://api.globalbioticinteractions.org/interaction.svg?interactionType=ecologicallyRelatedTo&accordingTo=globi:hurlbertlab/dietdatabase&refutes=true&refutes=false)](https://www.globalbioticinteractions.org/?interactionType=ecologicallyRelatedTo&accordingTo=globi:hurlbertlab/dietdatabase&refutes=true&refutes=false) 40 | [![GloBI Review](https://depot.globalbioticinteractions.org/reviews/hurlbertlab/dietdatabase/review.svg)](https://depot.globalbioticinteractions.org/reviews/hurlbertlab/dietdatabase/README.txt) 41 | 42 | The Avian Diet Database is configured to be indexed by https://globalbioticinteractions.org . 43 | -------------------------------------------------------------------------------- /archive/AvianDietDatabase_old.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/archive/AvianDietDatabase_old.txt -------------------------------------------------------------------------------- /archive/Olsen2015-AppendixS1.xls: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/archive/Olsen2015-AppendixS1.xls -------------------------------------------------------------------------------- /archive/Olsen2015-AppendixS2.xls: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/archive/Olsen2015-AppendixS2.xls -------------------------------------------------------------------------------- /archive/diet_compendia.txt: -------------------------------------------------------------------------------- 1 | Source Species Data? Entered? 2 | Beal 1907 House finch y y 3 | Beal 1907 Western tanager y y 4 | Beal 1907 Cliff swallow y y 5 | Beal 1907 Barn swallow y y 6 | Beal 1907 Violet-green swallow y y 7 | Beal 1907 Loggerhead shrike y y 8 | Beal 1907 Warbling vireo y y 9 | Beal 1907 Cassin's vireo y y 10 | Beal 1907 Hutton's vireo y y 11 | Beal 1907 Yellow-rumped warbler y y 12 | Beal 1907 Townsend's warbler y y 13 | Beal 1907 Common yellowthroat y y 14 | Beal 1907 Orange-crowned warbler y y 15 | Beal 1907 Golden pileolated warbler y y 16 | Beal 1907 Northern mockingbird y y 17 | Beal 1907 California thrasher y y 18 | Beal 1907 Bewick's wren y y 19 | Beal 1907 House wren y y 20 | Beal 1907 Marsh wren y y 21 | Beal 1907 Cactus wren y y 22 | Beal 1907 Brown creeper y y 23 | Beal 1907 Pygmy nuthatch y y 24 | Beal 1907 Plain titmouse y y 25 | Beal 1907 Chestnut-sided chickadee y y 26 | Beal 1907 Wrentit y y 27 | Beal 1907 Bushtit y y 28 | Beal 1907 Ruby-crowned kinglet y y 29 | Beal 1907 Golden-crowned kinglet y y 30 | Beal 1907 California gnatcatcher y y 31 | Beal 1907 Black-tailed gnatcatcher y y 32 | Beal 1907 Hermit thrush y y 33 | Beal 1907 American robin y y 34 | Beal 1907 Western bluebird y y 35 | Beal 1907 Swainson's thrush y y 36 | Beal 1910 California quail y y 37 | Beal 1910 Hairy woodpecker y y 38 | Beal 1910 Downy woodpecker y y 39 | Beal 1910 Nuttall's woodpecker y y 40 | Beal 1910 Red-breasted sapsucker y y 41 | Beal 1910 Acorn woodpecker y y 42 | Beal 1910 Northern flicker y y 43 | Beal 1910 Ash-throated flycatcher y y 44 | Beal 1910 Western kingbird y y 45 | Beal 1910 Cassin's kingbird y y 46 | Beal 1910 Say's phoebe y y 47 | Beal 1910 Black phoebe y y 48 | Beal 1910 Western wood-pewee y y 49 | Beal 1910 Western flycatcher y y 50 | Beal 1910 Horned lark y y 51 | Beal 1910 Steller's jay y y 52 | Beal 1910 Western scrub jay y y 53 | Beal 1910 Red-winged blackbird y y 54 | Beal 1910 Brewer's blackbird y y 55 | Beal 1910 Western meadowlark y y 56 | Beal 1910 Bullock's oriole y y 57 | Beal 1910 American goldfinch y y 58 | Beal 1910 Lesser goldfinch y y 59 | Beal 1910 White-crowned sparrow y y 60 | Beal 1910 Golden-crowned sparrow y y 61 | Beal 1910 Chipping sparrow y y 62 | Beal 1910 Song sparrow y y 63 | Beal 1910 Spotted towhee y y 64 | Beal 1910 California towhee y y 65 | Beal 1910 Black-headed grosbeak y y 66 | Beal 1911 Hairy woodpecker y y 67 | Beal 1911 Downy woodpecker y y 68 | Beal 1911 Red-cockaded woodpecker y y 69 | Beal 1911 Nuttall's woodpecker y y 70 | Beal 1911 Three-toed woodpecker y y 71 | Beal 1911 Yellow-bellied sapsucker y y 72 | Beal 1911 Red-breasted sapsucker y y 73 | Beal 1911 Williamson's sapsucker y y 74 | Beal 1911 Pileated woodpecker y y 75 | Beal 1911 Red-headed woodpecker y y 76 | Beal 1911 Acorn woodpecker y y 77 | Beal 1911 Lewis' woodpecker y y 78 | Beal 1911 Red-bellied woodpecker y y 79 | Beal 1911 Northern flicker y y 80 | Beal 1911 Ivory-billed woodpecker y y 81 | Beal 1912 Scissor-tailed flycatcher y y 82 | Beal 1912 Eastern kingbird y y 83 | Beal 1912 Western kingbird y y 84 | Beal 1912 Cassin's kingbird y y 85 | Beal 1912 Great crested flycatcher y y 86 | Beal 1912 Ash-throated flycatcher y y 87 | Beal 1912 Eastern phoebe y y 88 | Beal 1912 Say's phoebe y y 89 | Beal 1912 Black phoebe y y 90 | Beal 1912 Olive-sided flycatcher y y 91 | Beal 1912 Eastern wood-pewee y y 92 | Beal 1912 Western wood-pewee y y 93 | Beal 1912 Yellow-bellied flycatcher y y 94 | Beal 1912 Western flycatcher y y 95 | Beal 1912 Acadian flycatcher y y 96 | Beal 1912 Willow/Alder flycatcher y y 97 | Beal 1912 Least flycatcher y y 98 | Beal 1915 American robin y y 99 | Beal 1915 Varied thrush y y 100 | Beal 1915 Eastern bluebird y y 101 | Beal 1915 Western bluebird y y 102 | Beal 1915 Mountain bluebird y y 103 | Beal 1918 Purple martin y y 104 | Beal 1918 Cliff swallow y y 105 | Beal 1918 Barn swallow y y 106 | Beal 1918 Tree swallow y y 107 | Beal 1918 Violet-green swallow y y 108 | Beal 1918 Bank swallow y y 109 | Beal 1918 Northern rough-winged swallow 110 | Beal et al. 1941 House wren 111 | Beal et al. 1941 Carolina wren 112 | Beal et al. 1941 Northern mockingbird 113 | Beal et al. 1941 Brown thrasher 114 | Beal et al. 1941 Northern cardinal 115 | Beal et al. 1941 Pyrrhuloxia 116 | Beal et al. 1941 Painted bunting 117 | Beal et al. 1941 American crow 118 | Beal et al. 1941 Blue jay 119 | Beal et al. 1941 Eastern meadowlark 120 | Beal et al. 1941 Boat-tailed grackle 121 | Beal et al. 1941 Eastern bluebird 122 | Beal et al. 1941 Carolina chickadee 123 | Beal et al. 1941 Tufted titmouse 124 | Beal et al. 1941 Purple martin 125 | Beal et al. 1941 Scissor-tailed flycatcher 126 | Beal et al. 1941 Red-cockaded woodpecker 127 | Beal et al. 1941 Chuck-will's-widow 128 | Beal et al. 1941 Common nighthawk 129 | Beal et al. 1941 Yellow-billed cuckoo 130 | Beal et al. 1941 Northern bobwhite 131 | Beal et al. 1941 Swallow-tailed kite 132 | Beal et al. 1941 Turkey vulture 133 | -------------------------------------------------------------------------------- /archive/eBird_Taxonomy_v1.55.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/archive/eBird_Taxonomy_v1.55.txt -------------------------------------------------------------------------------- /archive/felder.tsv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/archive/felder.tsv -------------------------------------------------------------------------------- /birdtaxonomy/Clements_Taxonomy_v2018_NorthAmerica.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/birdtaxonomy/Clements_Taxonomy_v2018_NorthAmerica.csv -------------------------------------------------------------------------------- /birdtaxonomy/aba_to_ebird_taxonomy_conversion.csv: -------------------------------------------------------------------------------- 1 | common_name.aba,sci_name.aba,common_name.ebird,sci_name.ebird,code 2 | Dark-billed Cuckoo,Cuculus melacoryphus,Dark-billed Cuckoo,Coccyzus melacoryphus,5 3 | Fork-tailed Swift,Apus pacificus,Pacific Swift,Apus pacificus,4 4 | Paint-billed Crake,Neocrex erythrops,Paint-billed Crake,Mustelirallus erythrops,5 5 | Purple Gallinule,Porphyrio martinicus,Purple Gallinule,Porphyrio martinica,1 6 | Purple Swamphen,Porphyrio porphyrio,Western Swamphen,Porphyrio porphyrio,2 7 | Common Moorhen,Gallinula chloropus,Eurasian Moorhen,Gallinula chloropus,5 8 | Spoon-billed Sandpiper,Calidris pygmea,Spoon-billed Sandpiper,Calidris pygmaea,4 9 | Fork-tailed Storm-Petrel,Hydrobates furcatus,Fork-tailed Storm-Petrel,Oceanodroma furcata,2 10 | Ringed Storm-Petrel,Hydrobates hornbyi,Ringed Storm-Petrel,Oceanodroma hornbyi,5 11 | Swinhoe's Storm-Petrel,Hydrobates monorhis,Swinhoe's Storm-Petrel,Oceanodroma monorhis,5 12 | Leach's Storm-Petrel,Hydrobates leucorhous,Leach's Storm-Petrel,Oceanodroma leucorhoa,1 13 | Townsend's Storm-Petrel,Hydrobates socorroensis,Townsend's Storm-Petrel,Oceanodroma socorroensis,3 14 | Ashy Storm-Petrel,Hydrobates homochroa,Ashy Storm-Petrel,Oceanodroma homochroa,2 15 | Band-rumped Storm-Petrel,Hydrobates castro,Band-rumped Storm-Petrel,Oceanodroma castro,2 16 | Wedge-rumped Storm-Petrel,Hydrobates tethys,Wedge-rumped Storm-Petrel,Oceanodroma tethys,4 17 | Black Storm-Petrel,Hydrobates melania,Black Storm-Petrel,Oceanodroma melania,2 18 | Tristram's Storm-Petrel,Hydrobates tristrami,Tristram's Storm-Petrel,Oceanodroma tristrami,3 19 | Least Storm-Petrel,Hydrobates microsoma,Least Storm-Petrel,Oceanodroma microsoma,3 20 | Japanese Bush-Warbler,Horornis diphone,Japanese Bush Warbler,Horornis diphone,2 21 | Japanese White-eye,Zosterops japonicus,Warbling White-eye,Zosterops japonicus,2 22 | Greater Necklaced Laughingthrush,Garrulax pectoralis,Greater Necklaced Laughingthrush,Ianthocincla pectoralis,3 23 | Hwamei,Garrulax canorus,Chinese Hwamei,Garrulax canorus,2 24 | Thick-billed Warbler,Arudinax aedon,Thick-billed Warbler,Arundinax aedon,5 25 | Bluethroat,Cyanecula svecica,Bluethroat,Luscinia svecica,2 26 | Stonechat,Saxicola torquatus,African Stonechat,Saxicola torquatus,4 27 | Dusky Thrush,Turdus naumanni,Dusky Thrush,Turdus eunomus,4 28 | -------------------------------------------------------------------------------- /birdtaxonomy/birdies.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/birdtaxonomy/birdies.csv -------------------------------------------------------------------------------- /birdtaxonomy/creating_NA_specieslist.r: -------------------------------------------------------------------------------- 1 | # Get a species list of North American birds based on the ABA checklist, 2 | # and merge in families and orders from NACC (AOU). 3 | 4 | # This requires manually cleaning/troubleshooting names that do not match between lists. 5 | # This has been done and is included in the aba_to_ebird_taxonomy_conversion.csv, 6 | # which might need to be updated with future list releases. 7 | 8 | library(tidyverse) 9 | library(devtools) 10 | library(stringr) 11 | 12 | aba_ebird = read_csv('birdtaxonomy/aba_to_ebird_taxonomy_conversion.csv', col_names = T, quote = '\"') 13 | 14 | ebird = read_csv('birdtaxonomy/eBird_Taxonomy_v2019.csv', col_names = T, quote = '\"') %>% 15 | rename(common_name = PRIMARY_COM_NAME, sci_name = SCI_NAME, order = ORDER1) %>% 16 | mutate(family = word(FAMILY, 1)) %>% 17 | filter(CATEGORY == 'species') %>% 18 | select(common_name, sci_name, family, order) 19 | 20 | aba = read_csv('birdtaxonomy/ABA_Checklist-8.0.6a.csv', skip = 2, quote = '\"', col_names = FALSE) %>% 21 | rename(common_name = X2, sci_name = X3, code = X5) %>% 22 | select(common_name, sci_name, code) %>% 23 | filter(!is.na(common_name), 24 | code <= 3) # ABA Rarity Code of 1, 2, or 3 (i.e., exclude mega-rarities of code 4 and 5) 25 | 26 | matched = aba %>% 27 | inner_join(ebird, by = c('common_name', 'sci_name')) 28 | 29 | unmatched = aba_ebird %>% 30 | inner_join(ebird, by = c('common_name.ebird' = 'common_name', 'sci_name.ebird' = 'sci_name')) %>% 31 | rename(common_name = common_name.ebird, sci_name = sci_name.ebird) %>% 32 | filter(code <= 3) %>% 33 | select(common_name, sci_name, code, family, order) 34 | 35 | NA_specieslist = bind_rows(matched, unmatched) %>% 36 | arrange(order, family, sci_name) 37 | 38 | write.csv(NA_specieslist, 'birdtaxonomy/NA_specieslist.csv', row.names = F) 39 | -------------------------------------------------------------------------------- /birdtaxonomy/eBird-Clements-v2021-integrated-checklist-August-2021.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/birdtaxonomy/eBird-Clements-v2021-integrated-checklist-August-2021.csv -------------------------------------------------------------------------------- /birdtaxonomy/eBird_Taxonomy_v2016.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/birdtaxonomy/eBird_Taxonomy_v2016.csv -------------------------------------------------------------------------------- /birdtaxonomy/eBird_Taxonomy_v2016_NorthAmerica.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/birdtaxonomy/eBird_Taxonomy_v2016_NorthAmerica.csv -------------------------------------------------------------------------------- /birdtaxonomy/eBird_Taxonomy_v2018.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/birdtaxonomy/eBird_Taxonomy_v2018.csv -------------------------------------------------------------------------------- /birdtaxonomy/eBird_Taxonomy_v2019.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/birdtaxonomy/eBird_Taxonomy_v2019.csv -------------------------------------------------------------------------------- /birdtaxonomy/ebird_taxonomy_v2021_subset.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/birdtaxonomy/ebird_taxonomy_v2021_subset.csv -------------------------------------------------------------------------------- /birdtaxonomy/orders.csv: -------------------------------------------------------------------------------- 1 | Order,Family 2 | Struthioniformes,Struthionidae 3 | Rheiformes,Rheidae 4 | Tinamiformes,Tinamidae 5 | Casuariiformes,Casuariidae 6 | Apterygiformes,Apterygidae 7 | Anseriformes,Anhimidae 8 | Anseriformes,Anseranatidae 9 | Anseriformes,Anatidae 10 | Galliformes,Megapodiidae 11 | Galliformes,Cracidae 12 | Galliformes,Numididae 13 | Galliformes,Odontophoridae 14 | Galliformes,Phasianidae 15 | Gaviiformes,Gaviidae 16 | Podicipediformes,Podicipedidae 17 | Phoenicopteriformes,Phoenicopteridae 18 | Sphenisciformes,Spheniscidae 19 | Procellariiformes,Diomedeidae 20 | Procellariiformes,Procellariidae 21 | Procellariiformes,Hydrobatidae 22 | Phaethontiformes,Phaethontidae 23 | Ciconiiformes,Ciconiidae 24 | Suliformes,Fregatidae 25 | Suliformes,Sulidae 26 | Suliformes,Phalacrocoracidae 27 | Suliformes,Anhingidae 28 | Pelecaniformes,Pelecanidae 29 | Pelecaniformes,Balaenicipitidae 30 | Pelecaniformes,Scopidae 31 | Pelecaniformes,Ardeidae 32 | Pelecaniformes,Threskiornithidae 33 | Accipitriformes,Cathartidae 34 | Accipitriformes,Sagittariidae 35 | Accipitriformes,Pandionidae 36 | Accipitriformes,Accipitridae 37 | Otidiformes,Otididae 38 | Mesitornithiformes,Mesitornithidae 39 | Eurypygiformes,Rhynochetidae 40 | Eurypygiformes,Eurypygidae 41 | Gruiformes,Rallidae 42 | Gruiformes,Sarothruridae 43 | Gruiformes,Heliornithidae 44 | Gruiformes,Aramidae 45 | Gruiformes,Psophiidae 46 | Gruiformes,Gruidae 47 | Charadriiformes,Chionidae 48 | Charadriiformes,Pluvianellidae 49 | Charadriiformes,Burhinidae 50 | Charadriiformes,Pluvianidae 51 | Charadriiformes,Recurvirostridae 52 | Charadriiformes,Ibidorhynchidae 53 | Charadriiformes,Haematopodidae 54 | Charadriiformes,Charadriidae 55 | Charadriiformes,Pedionomidae 56 | Charadriiformes,Thinocoridae 57 | Charadriiformes,Rostratulidae 58 | Charadriiformes,Jacanidae 59 | Charadriiformes,Scolopacidae 60 | Charadriiformes,Turnicidae 61 | Charadriiformes,Dromadidae 62 | Charadriiformes,Glareolidae 63 | Charadriiformes,Stercorariidae 64 | Charadriiformes,Alcidae 65 | Charadriiformes,Laridae 66 | Pterocliformes,Pteroclidae 67 | Columbiformes,Columbidae 68 | Opisthocomiformes,Opisthocomidae 69 | Cuculiformes,Musophagidae 70 | Cuculiformes,Cuculidae 71 | Strigiformes,Tytonidae 72 | Strigiformes,Strigidae 73 | Caprimulgiformes,Podargidae 74 | Caprimulgiformes,Caprimulgidae 75 | Caprimulgiformes,Nyctibiidae 76 | Caprimulgiformes,Steatornithidae 77 | Caprimulgiformes,Aegothelidae 78 | Caprimulgiformes,Apodidae 79 | Caprimulgiformes,Hemiprocnidae 80 | Caprimulgiformes,Trochilidae 81 | Coliiformes,Coliidae 82 | Leptosomiformes,Leptosomidae 83 | Trogoniformes,Trogonidae 84 | Bucerotiformes,Upupidae 85 | Bucerotiformes,Phoeniculidae 86 | Bucerotiformes,Bucorvidae 87 | Bucerotiformes,Bucerotidae 88 | Coraciiformes,Todidae 89 | Coraciiformes,Momotidae 90 | Coraciiformes,Alcedinidae 91 | Coraciiformes,Meropidae 92 | Coraciiformes,Coraciidae 93 | Coraciiformes,Brachypteraciidae 94 | Galbuliformes,Bucconidae 95 | Galbuliformes,Galbulidae 96 | Piciformes,Lybiidae 97 | Piciformes,Megalaimidae 98 | Piciformes,Capitonidae 99 | Piciformes,Semnornithidae 100 | Piciformes,Ramphastidae 101 | Piciformes,Indicatoridae 102 | Piciformes,Picidae 103 | Cariamiformes,Cariamidae 104 | Falconiformes,Falconidae 105 | Psittaciformes,Strigopidae 106 | Psittaciformes,Cacatuidae 107 | Psittaciformes,Psittaculidae 108 | Psittaciformes,Psittacidae 109 | Passeriformes,Acanthisittidae 110 | Passeriformes,Calyptomenidae 111 | Passeriformes,Eurylaimidae 112 | Passeriformes,Sapayoidae 113 | Passeriformes,Philepittidae 114 | Passeriformes,Pittidae 115 | Passeriformes,Thamnophilidae 116 | Passeriformes,Melanopareiidae 117 | Passeriformes,Conopophagidae 118 | Passeriformes,Grallariidae 119 | Passeriformes,Rhinocryptidae 120 | Passeriformes,Formicariidae 121 | Passeriformes,Furnariidae 122 | Passeriformes,Tyrannidae 123 | Passeriformes,Oxyruncidae 124 | Passeriformes,Cotingidae 125 | Passeriformes,Pipridae 126 | Passeriformes,Tityridae 127 | Passeriformes,Menuridae 128 | Passeriformes,Atrichornithidae 129 | Passeriformes,Ptilonorhynchidae 130 | Passeriformes,Climacteridae 131 | Passeriformes,Maluridae 132 | Passeriformes,Meliphagidae 133 | Passeriformes,Dasyornithidae 134 | Passeriformes,Pardalotidae 135 | Passeriformes,Acanthizidae 136 | Passeriformes,Pomatostomidae 137 | Passeriformes,Orthonychidae 138 | Passeriformes,Cnemophilidae 139 | Passeriformes,Melanocharitidae 140 | Passeriformes,Mohouidae 141 | Passeriformes,Paramythiidae 142 | Passeriformes,Callaeidae 143 | Passeriformes,Notiomystidae 144 | Passeriformes,Psophodidae 145 | Passeriformes,Cinclosomatidae 146 | Passeriformes,Platysteiridae 147 | Passeriformes,Vangidae 148 | Passeriformes,Malaconotidae 149 | Passeriformes,Machaerirhynchidae 150 | Passeriformes,Artamidae 151 | Passeriformes,Cracticidae 152 | Passeriformes,Pityriaseidae 153 | Passeriformes,Aegithinidae 154 | Passeriformes,Campephagidae 155 | Passeriformes,Neosittidae 156 | Passeriformes,Eulacestomatidae 157 | Passeriformes,Falcunculidae 158 | Passeriformes,Pachycephalidae 159 | Passeriformes,Rhagologidae 160 | Passeriformes,Oreoicidae 161 | Passeriformes,Laniidae 162 | Passeriformes,Vireonidae 163 | Passeriformes,Oriolidae 164 | Passeriformes,Dicruridae 165 | Passeriformes,Rhipiduridae 166 | Passeriformes,Ifritidae 167 | Passeriformes,Monarchidae 168 | Passeriformes,Corvidae 169 | Passeriformes,Corcoracidae 170 | Passeriformes,Paradisaeidae 171 | Passeriformes,Melampittidae 172 | Passeriformes,Petroicidae 173 | Passeriformes,Picathartidae 174 | Passeriformes,Chaetopidae 175 | Passeriformes,Eupetidae 176 | Passeriformes,Panuridae 177 | Passeriformes,Nicatoridae 178 | Passeriformes,Alaudidae 179 | Passeriformes,Hirundinidae 180 | Passeriformes,Stenostiridae 181 | Passeriformes,Paridae 182 | Passeriformes,Remizidae 183 | Passeriformes,Aegithalidae 184 | Passeriformes,Sittidae 185 | Passeriformes,Tichodromidae 186 | Passeriformes,Certhiidae 187 | Passeriformes,Troglodytidae 188 | Passeriformes,Polioptilidae 189 | Passeriformes,Cinclidae 190 | Passeriformes,Pycnonotidae 191 | Passeriformes,Regulidae 192 | Passeriformes,Pnoepygidae 193 | Passeriformes,Macrosphenidae 194 | Passeriformes,Cettiidae 195 | Passeriformes,Phylloscopidae 196 | Passeriformes,Acrocephalidae 197 | Passeriformes,Locustellidae 198 | Passeriformes,Donacobiidae 199 | Passeriformes,Bernieridae 200 | Passeriformes,Cisticolidae 201 | Passeriformes,Sylviidae 202 | Passeriformes,Paradoxornithidae 203 | Passeriformes,Zosteropidae 204 | Passeriformes,Timaliidae 205 | Passeriformes,Pellorneidae 206 | Passeriformes,Leiothrichidae 207 | Passeriformes,Promeropidae 208 | Passeriformes,Irenidae 209 | Passeriformes,Hyliotidae 210 | Passeriformes,Muscicapidae 211 | Passeriformes,Turdidae 212 | Passeriformes,Mimidae 213 | Passeriformes,Sturnidae 214 | Passeriformes,Buphagidae 215 | Passeriformes,Chloropseidae 216 | Passeriformes,Dicaeidae 217 | Passeriformes,Nectariniidae 218 | Passeriformes,Prunellidae 219 | Passeriformes,Motacillidae 220 | Passeriformes,Urocynchramidae 221 | Passeriformes,Elachuridae 222 | Passeriformes,Bombycillidae 223 | Passeriformes,Mohoidae 224 | Passeriformes,Ptiliogonatidae 225 | Passeriformes,Dulidae 226 | Passeriformes,Hylocitreidae 227 | Passeriformes,Hypocoliidae 228 | Passeriformes,Peucedramidae 229 | Passeriformes,Calcariidae 230 | Passeriformes,Parulidae 231 | Passeriformes,Thraupidae 232 | Passeriformes,Emberizidae 233 | Passeriformes,Cardinalidae 234 | Passeriformes,Icteridae 235 | Passeriformes,Fringillidae 236 | Passeriformes,Passeridae 237 | Passeriformes,Ploceidae 238 | Passeriformes,Estrildidae 239 | Passeriformes,Viduidae 240 | -------------------------------------------------------------------------------- /cleaning/AvianDietDatabase_unstandardized_testfile.txt: -------------------------------------------------------------------------------- 1 | Common_Name Scientific_Name Subspecies Family Taxonomy Longitude_dd Latitude_dd Altitude_min_m Altitude_mean_m Altitude_max_m Location_Region Location_Specific Habitat_type Observation_Month_Begin Observation_Year_Begin Observation_Month_End Observation_Year_End Observation_Season Analysis_Number Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Inclusive_Prey_Taxon Prey_Name_ITIS_ID Prey_Name_Status Prey_Stage Prey_Part Prey_Common_Name Fraction_Diet Diet_Type Item_Sample_Size Bird_Sample_Size Sites Study_Type Notes Entered_By Source 2 | Black Francolin Francolinus francolinus Phasianidae Clements Taxonomy v2018 NA NA NA NA NA Louisiana "Gum Cove, Cameron Parish " wetland; agriculture; grassland 1 1967 2 1967 winter 1 Animalia Arthropoda Insecta Diptera Tipulidae yes larva 0.1 occurrence 6 1 emetic EK "Palermo, R. J. and G. L. Doster. 1971. A comparison of the late winter foods and parasites of Bobwhite Quail and Black Francolin in southwestern Louisiana. Proc. of the 24th Annual Conf. Southeastern Assoc. of Game & Fish Commissioners, Sept. 27-30 1970 (Webb, J. W., Ed.) Atlanta, GA." 3 | Black Francolin francolinus francolinus Phasianidae Clements Taxonomy v2018 NA NA NA NA NA Louisiana "Gum Cove, Cameron Parish " wetland; agriculture; grassland 1 1967 2 1967 winter 1 Animalia Arthropoda Insecta Coleoptera Chrysomelidae Diabrotica Diabrotica undecimpunctata Larva 0.1 occurrence 6 1 Emetic EK "Palermo, R. J. and G. L. Doster. 1971. A comparison of the late winter foods and parasites of Bobwhite Quail and Black Francolin in southwestern Louisiana. Proc. of the 24th Annual Conf. Southeastern Assoc. of Game & Fish Commissioners, Sept. 27-30 1970 (Webb, J. W., Ed.) Atlanta, GA." 4 | Black Francolin Francolinus francolinus Phasianidae Clements Taxonomy v2018 NA NA NA NA NA Louisiana "Gum Cove, Cameron Parish " Wetland; Agriculture; Grassland 1 1967 2 1967 winter 1 Animalia Arthropoda Insecta Hymenoptera Formicidae Adult 0.1 occurrence 6 1 stomach contents EK "Palermo, R. J. and G. L. Doster. 1971. A comparison of the late winter foods and parasites of Bobwhite Quail and Black Francolin in southwestern Louisiana. Proc. of the 24th Annual Conf. Southeastern Assoc. of Game & Fish Commissioners, Sept. 27-30 1970 (Webb, J. W., Ed.) Atlanta, GA." 5 | Black Francolin Francolinus francolinus Phasianidae Clements Taxonomy v2018 NA NA NA NA NA Louisiana "Gum Cove, Cameron Parish " Wetland; Agriculture; Grassland 1 1967 2 1967 winter 1 Animalia Arthropoda Insecta Lepidoptera Bud 0.1 occurrence 6 1 Stomach contents EK "Palermo, R. J. and G. L. Doster. 1971. A comparison of the late winter foods and parasites of Bobwhite Quail and Black Francolin in southwestern Louisiana. Proc. of the 24th Annual Conf. Southeastern Assoc. of Game & Fish Commissioners, Sept. 27-30 1970 (Webb, J. W., Ed.) Atlanta, GA." 6 | Black Francolin Francolinus francolinus Phasianidae Clements Taxonomy v2018 NA NA NA NA NA Louisiana "Gum Cove, Cameron Parish " Wetland; Agriculture; Grassland 1 1967 2 1967 winter 1 Animalia Arthropoda Insecta Hemiptera 0.1 Occurrence 6 1 DNA sequencing EK "Palermo, R. J. and G. L. Doster. 1971. A comparison of the late winter foods and parasites of Bobwhite Quail and Black Francolin in southwestern Louisiana. Proc. of the 24th Annual Conf. Southeastern Assoc. of Game & Fish Commissioners, Sept. 27-30 1970 (Webb, J. W., Ed.) Atlanta, GA." 7 | Black francolin Francolinus francolinus Phasianidae Clements Taxonomy v2018 NA NA NA NA NA Louisiana "Gum Cove, Cameron Parish " wetland; agriculture; grassland 1 1967 2 1967 winter 1 Animalia Arthropoda Insecta Psocoptera 0.2 Wt_or_Vol 6 1 dna sequencing EK "Palermo, R. J. and G. L. Doster. 1971. A comparison of the late winter foods and parasites of Bobwhite Quail and Black Francolin in southwestern Louisiana. Proc. of the 24th Annual Conf. Southeastern Assoc. of Game & Fish Commissioners, Sept. 27-30 1970 (Webb, J. W., Ed.) Atlanta, GA." 8 | Black Francolin Francolinus francolinus Phasianidae Clements Taxonomy v2018 NA NA NA NA NA Louisiana "Gum Cove, Cameron Parish " wetland; agriculture; grassland 1 1967 2 1967 Winter 1 Animalia Arthropoda Insecta Orthoptera 0.2 Wt_Or_Vol 6 1 EK "Palermo, R. J. and G. L. Doster. 1971. A comparison of the late winter foods and parasites of Bobwhite Quail and Black Francolin in southwestern Louisiana. Proc. of the 24th Annual Conf. Southeastern Assoc. of Game & Fish Commissioners, Sept. 27-30 1970 (Webb, J. W., Ed.) Atlanta, GA." 9 | Black Francolin Francolinus francolinus Phasianidae Clements Taxonomy v2018 NA NA NA NA NA Louisiana "Gum Cove, Cameron Parish " wetland; agriculture; grassland 1 1967 2 1967 Winter 1 Animalia Arthropoda Insecta Plecoptera 0.2 Wt_or_vol 6 1 EK "Palermo, R. J. and G. L. Doster. 1971. A comparison of the late winter foods and parasites of Bobwhite Quail and Black Francolin in southwestern Louisiana. Proc. of the 24th Annual Conf. Southeastern Assoc. of Game & Fish Commissioners, Sept. 27-30 1970 (Webb, J. W., Ed.) Atlanta, GA." 10 | Black Francolin Francolinus francolinus Phasianidae Clements Taxonomy v2018 NA NA NA NA NA Louisiana "Gum Cove, Cameron Parish " wetland; agriculture; grassland 1 1967 2 1967 Winter 1 Animalia Arthropoda Insecta hymenoptera apidae 0.2 Wt_or_Vol 6 1 EK "Palermo, R. J. and G. L. Doster. 1971. A comparison of the late winter foods and parasites of Bobwhite Quail and Black Francolin in southwestern Louisiana. Proc. of the 24th Annual Conf. Southeastern Assoc. of Game & Fish Commissioners, Sept. 27-30 1970 (Webb, J. W., Ed.) Atlanta, GA." 11 | Black Francolin Francolinus francolinus Phasianidae Clements Taxonomy v2018 NA NA NA NA NA Louisiana "Gum Cove, Cameron Parish " wetland; agriculture; grassland 1 1967 2 1967 Winter 1 Animalia Arthropoda Insecta Ephemeroptera 0.2 Wt_or_Vol 6 1 EK "Palermo, R. J. and G. L. Doster. 1971. A comparison of the late winter foods and parasites of Bobwhite Quail and Black Francolin in southwestern Louisiana. Proc. of the 24th Annual Conf. Southeastern Assoc. of Game & Fish Commissioners, Sept. 27-30 1970 (Webb, J. W., Ed.) Atlanta, GA." 12 | -------------------------------------------------------------------------------- /cleaning/Database_notes_and_errors.md: -------------------------------------------------------------------------------- 1 | Error Checking the Diet Database 2 | ================================ 3 | 4 | # General 5 | Early papers do not have standardized habitat descriptions 6 | Unidentified yes/no is not consistent 7 | Verifying prey names stops midway through database 8 | 9 | 10 | # Beal 1898 11 | Data from yellow-billed and black-billed cuckoos 12 | Number of sites in paper seems to be unclear, not necessarily 999 13 | 14 | 15 | # Beal 1900 16 | Some data is entered about Brewer's blackbird early in database, but more relevant data has been added so initial data maybe should be deleted 17 | 18 | 19 | # Beal 1912 20 | Habitat descriptions need to be standardized 21 | Of the data listed from this study, only the following sets of data from these species add up to 1: 22 | Ash-throated flycatcher, diet by items 23 | Cassin's kingbird, diet by items 24 | Say's phoebe, diet by items 25 | Scissor-tailed flycatcher, diet by items 26 | Great crested flycatcher, diet unspecified 27 | 28 | 29 | # Beal 1918 30 | Fraction diet by occurrence adds up to 2 31 | 32 | 33 | # Beal 1941 34 | Number of sites in paper seems to be unclear, not necessarily 999 35 | 36 | 37 | # Bent 1942 38 | Habitat descriptions need to be standardized 39 | 40 | # Carlisle 2012 41 | Data includes frequency by item for family when available; not all frequencies for each bird species add up to 1 in original data 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | -------------------------------------------------------------------------------- /cleaning/Olsen_db_cleaning_history.r: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/cleaning/Olsen_db_cleaning_history.r -------------------------------------------------------------------------------- /cleaning/beal_cleaning.R: -------------------------------------------------------------------------------- 1 | install.packages(c('dplyr', 'stringr', 'tidyr')) 2 | 3 | setwd('C:/git/dietdatabase') 4 | source('scripts/database_summary_functions.R') 5 | 6 | diet = read.table('aviandietdatabase.txt', header=T, sep = '\t', quote = '\"', 7 | fill=T, stringsAsFactors = F) 8 | 9 | 10 | beal = unique(diet[grep("Beal", diet$Source), c('Common_Name', 'Source')]) %>% 11 | arrange(Common_Name, Source) 12 | 13 | beal$study = substr(beal$Source, 1, 19) 14 | 15 | beal[, c(1,3)] 16 | 17 | -------------------------------------------------------------------------------- /cleaning/cleaning_overview.md: -------------------------------------------------------------------------------- 1 | # Cleaning the Avian Diet Database 2 | 3 | There are various sources of error that may creep into the database, and we will need to employ several 4 | different strategies to catch and fix them. 5 | 6 | In R or RStudio, open the RProject file in the main repository, source the cleaning functions, and read in the file to be cleaned (e.g. 7 | AvianDietDatabase_Beaver_and_Baldwin_1975.txt). 8 | 9 | ``` 10 | ### You'll want to replace the 'qa_qc_test_db.txt' file with the data file you want to clean. 11 | source('cleaning/database_error_checking.R') 12 | dietdb = read.table('cleaning/qa_qc_test_db.txt', header = T, sep = '\t', quote = '\"', stringsAsFactors = F) 13 | ``` 14 | 15 | In the example above we read in a test file, 'qa_qc_test_db.txt', that we know has several specific errors: 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
Row #Problem
1Location_Region is 'Rode Island', a typo which does not match currently accepted names
2Habitat_type is 'scrubland', which does not match currently accepted names (should be 'shrubland')
3Longitude is not within (-180, 180)
4Prey_Stage is 'juvie', which does not match currently accepted names (should be 'juvenile')
5Location_Specific is 'Multipe', a typo of 'Multiple'
7Prey_Part is 'fruity', which does not match currently accepted names (should be 'fruit')
8Scientific_Name, 'Melospiza georgina', does not match any names in the most recent eBird Clements checklist
9Observation_Month_Begin is the text string 'August' rather than an integer indicating month number.
10Fraction_Diet is greater than 1
10Taxonomy is different from the most recent bird taxonomy, eBird Clements Checklist v2016
63 | 64 | 65 | ## Overall database summary 66 | To get a sense of the total number of records, species covered, etc of the database sample you are cleaning, use the 67 | function `dbSummary()` like this: 68 | ``` 69 | > dbSummary(dietdb) 70 | $numRecords 71 | [1] 12 72 | 73 | $numSpecies 74 | [1] 3 75 | 76 | $numStudies 77 | [1] 1 78 | 79 | $recordsPerSpecies 80 | Common_Name n 81 | 1 Eastern Towhee 4 82 | 2 Swamp Sparrow 4 83 | 3 White-throated Sparrow 4 84 | 85 | $speciesPerFamily 86 | ORDER Family SpeciesWithData WithoutData 87 | 1 Accipitriformes Accipitridae 0 24 88 | 2 Accipitriformes Cathartidae 0 3 89 | 3 Anseriformes Anatidae 0 41 90 | 4 Caprimulgiformes Apodidae 0 4 91 | 5 Caprimulgiformes Caprimulgidae 0 6 92 | 6 Caprimulgiformes Trochilidae 0 14 93 | 7 Charadriiformes Alcidae 0 17 94 | 8 Charadriiformes Charadriidae 0 6 95 | ... 96 | ``` 97 | 98 | 99 | ## QA/QC: Checking for outliers, typos, and invalid values. 100 | We use the 'qa_qc()' function to conduct a basic check to catch any obvious errors. The purpose of this 101 | QA/QC check is to standardize the case (e.g. upper vs lower) of records, trim accidental leading or trailing whitespace in text strings, and to point out records that should be double-checked or possibly corrected. Let's work through 102 | the printed results. 103 | 104 | By default this function will write a new file using a name you specify with automatically standardized text fields. It is this new file that you will follow up with to correct any further errors that were identified. 105 | ``` 106 | > qa_qc(dietdb, write = TRUE, filename = 'cleaned_dietdb.txt', fracsum_accuracy = 0.03) 107 | Problems were identified in some of the bird names as well as in the following fields: Taxonomy, Habitat_type, Prey_Stage, Prey_Part, Location_Region . Refer to the output below (or in the saved object) for details. Pay particular attention to any fields in the QA/QC output that are not 'OK'. A cleaned database file with standardized upper/lower cases and trimmed white space was saved as 'cleaned_dietdb.txt'. Be sure to use this file rather than the original for further cleaning. 108 | 109 | $Problem_bird_names 110 | Common_Name Scientific_Name Family 111 | 1 Eastern Towhee Pipilo erythrophthalmus Emberizidae 112 | 2 Swamp Sparrow Melospiza georgiana Emberizidae 113 | 3 Swamp Sparrow Melospiza georgina Emberizidae 114 | 4 White-throated Sparrow Zonotrichia albicollis Emberizidae 115 | ``` 116 | The `$Problem_bird_names` section highlights any species that either 1) have a typo or invalid name for either the common name, scientific 117 | name, or family name, or 2) have an error in the family assignment or scientific name assignment to the given common name. In this 118 | case, there is a typo in 'Melospiza georgina' (which should be 'georgiana'). Also, the name for the family of New World Sparrows has changed from 'Emberizidae' to Passerellidae', so all of these sparrow species are being flagged. 119 | 120 | ``` 121 | $Taxonomy 122 | Taxonomy n 123 | 1 eBird Clements Checklist v2015 1 124 | 2 eBird Clements Checklist v2016 11 125 | ``` 126 | Non-name based text fields are checked against accepted values. If no problems are detected then the field is "OK". Otherwise, 127 | a table of unaccepted names (or possible typos) and their frequency are provided. Bird species names should use the most recent eBird Clements taxonomy, and in this case out of date taxonomies have been used. 128 | 129 | ``` 130 | $Longitude_dd 131 | [1] 3 132 | 133 | $Latitude_dd 134 | [1] "OK" 135 | 136 | $Altitude_min_m 137 | [1] "All values NA" 138 | 139 | $Altitude_mean_m 140 | [1] "All values NA" 141 | 142 | $Altitude_max_m 143 | [1] "All values NA" 144 | 145 | ``` 146 | Numeric fields in the database are checked for any values that are suspicious or invalid. If all is well, output is simply "OK" 147 | for that field. Otherwise, the row number(s) of the flagged values are provided. In this case, row 3 has a problematic longitude value, 148 | while the Latitude field is ok. The Altitude fields are all NA, but this is ok and nothing needs to be done for those. 149 | 150 | ``` 151 | $Location_Region 152 | Location_Region n 153 | 1 Rode Island 1 154 | ``` 155 | In this case, the typo 'Rode Island' was noted as occurring once, and this should be corrected in the database. If a phrase comes up 156 | here that you think should be a valid region name, then post an issue on Github, and we will revise the list of accepted names if appropriate. 157 | 158 | ``` 159 | $Location_Specific 160 | Location_Specific n 161 | 1 Multiple 11 162 | 2 Multipe 1 163 | ``` 164 | There are too many possible Location_Specific names and those names are expected to be highly variable so we do not have a list of 165 | accepted possibilities to check against. As such, _this is the one field for which ALL values are listed along with their frequency 166 | in the database._ Using this table, we should be able to standardize these names within each database sample and catch simple typos 167 | like 'Multipe' instead of 'Multiple'. 168 | 169 | ``` 170 | $Observation_Season 171 | [1] "OK" 172 | 173 | $Habitat_type 174 | Habitat_type n 175 | 1 scrubland 1 176 | ``` 177 | 'Scrubland' did not match our list of accepted habitat types. In this case, it should be replaced by 'shrubland'. 178 | 179 | ``` 180 | $Observation_Month_Begin 181 | [1] "Field has non-numeric or non-integer values" 182 | 183 | $Observation_Year_Begin 184 | [1] "OK" 185 | 186 | $Observation_Month_End 187 | [1] "OK" 188 | 189 | $Observation_Year_End 190 | [1] "OK" 191 | 192 | $Prey_Stage 193 | Prey_Stage n 194 | 1 juvie 1 195 | 196 | $Prey_Part 197 | Prey_Part n 198 | 1 fruity 1 199 | ``` 200 | Observation month and year fields are ok, except for Observation_Month_Begin which should be a value from 1 to 12, but apparently has some non-numeric values. Upon inspection, we can see that someone typed 'August' instead of using the number 8. 'juvie' is an unacceptable Prey_Stage name, and 'fruity' is an unacceptable 201 | Prey_Part name. Both values should be fixed or replaced as appropriate (e.g. 'juvenile', 'fruit'). 202 | 203 | ``` 204 | $Fraction_Diet 205 | [1] 9 10 206 | ``` 207 | Rows 9 and 10 have Fraction_Diet values that are outside of the range 0-1. In this case, someone probably forgot to convert 208 | %s to fractions. 209 | 210 | ``` 211 | $Diet_Type 212 | [1] "OK" 213 | 214 | $Item_Sample_Size 215 | [1] "OK" 216 | 217 | $Bird_Sample_Size 218 | [1] "OK" 219 | 220 | $Sites 221 | [1] "OK" 222 | 223 | $Study_Type 224 | [1] "OK" 225 | ``` 226 | No errors in these fields. 227 | 228 | ``` 229 | $Fraction_sum_check 230 | Source Common_Name Observation_Year_Begin Observation_Month_Begin Observation_Season Bird_Sample_Size Habitat_type 231 | 1 test White-throated Sparrow 1993 8 fall 38 shrubland 232 | 2 test White-throated Sparrow 1993 8 summer 38 shrubland 233 | Location_Region Item_Sample_Size Diet_Type Sum_Diet 234 | 1 Rhode Island NA Wt_or_Vol 176.300 235 | 2 Rhode Island NA Wt_or_Vol 1.031 236 | ``` 237 | The final check is whether, for each diet analysis (i.e. combination of study, bird species, date, location, and habitat), the 238 | diet values of the different prey in the analysis sum to close to 1. Summing to 1 is only expected for Wt_or_Vol, Items, or 239 | Unspecified diet types, but not for Occurrence data. The `fracsum_accuracy` argument when calling `qa_qc()` specifies how 240 | close to 1 that sum should be. In this example, it was set to 0.03, which means any analyses where the sum of diet fractions is 241 | <0.97 or >1.03 will be listed here. 242 | 243 | In the first study , the Sum_Diet value is 176.3, way above 1. Something is clearly wrong with the Fraction_Diet data entered 244 | for this study. As we already noted above, someone probably entered %s instead of fractions, but even after that is corrected, 245 | this will still return 1.76. In this case, the possibilities are a typo in the values entered (so look up the original paper 246 | and check that they were entered correctly), or that the Diet_Type should actually be Occurrence instead of Wt_or_Vol, in which 247 | case a sum greater than 1 is ok. 248 | 249 | In the second study listed, you can see that the Sum_Diet is 1.021. This may reflect a small typo in the Fraction_Diet values 250 | entered (so compare values to the original paper), but it is so small it could also just reflect the accumulation of rounding 251 | errors. *I recommend setting `fracsum_accuracy = 0.03` as the default to minimize these types of false positives. 252 | 253 | **If the values in a reported study have been double-checked and do not sum to a value between 0.97 and 1.03, please enter this exact phrase in the Notes field: 254 | "values as reported do not sum to 100%".** (This can come before or after any other notes that need to be specified, separated by a semi-colon.) 255 | 256 | ## Cleaning taxonomic names of prey 257 | This is a big task, so we've got a separate page on it [here](https://github.com/hurlbertlab/dietdatabase/blob/master/cleaning/name_cleaning_instructions.md). 258 | 259 | ## Incorporating cleaned database records into main database 260 | One all typos, outliers, unaccepted values, and invalid taxonomic names have been corrected for the study you have entered, 261 | you may now incorporate these records into the main Avian Diet Database. The easiest way to do this is to 262 | 1) make sure you have the latest version of all files by typing `git pull origin master` in Git, 263 | 2) open your cleaned file (e.g. 'AvianDietDatabase_Beaver_and_Baldwin_1975_clean.txt'), 264 | 3) copy everything EXCEPT the header row, 265 | 4) open the main database file ('AvianDietDatabase.txt'), 266 | 5) paste the new cleaned records at the bottom and save the file, 267 | 6) commit this change in commit like `git commit -am "adding cleaned records from Beaver & Baldwin 1975"` 268 | 7) push these changes to the master repo: `git push origin master` 269 | 8) now you can delete all of the temporary files associated with that paper that has been entered (e.g. the original file, the 'clean' and 'badnames' files) 270 | 9) and use `git rm` to remove them so that git is no longer trying to track them. E.g., 271 | `git rm AvianDietDatabase_Beaver_and_Baldwin_1975.txt` for all the files you are deleting, followed at the end by 272 | `git commit -am "removing unneeded files"` , and finally `git push origin master` 273 | 274 | -------------------------------------------------------------------------------- /cleaning/compare_previous_db.R: -------------------------------------------------------------------------------- 1 | # Getting list of names that don't match between the current database and 2 | # an old version. (Apr 13 2017 seems to be last time cleaning was done) 3 | 4 | # Read in datasets 5 | diet = read.table('aviandietdatabase.txt', header=T, sep = '\t', quote = '\"', 6 | fill=T, stringsAsFactors = F) 7 | 8 | olddiet = read.table('dietdatabaseApr13.txt', header=T, sep = '\t', quote = '\"', 9 | fill=T, stringsAsFactors = F) 10 | olddiet2 = read.table('dietdatabaseJun22.txt', header=T, sep = '\t', quote = '\"', 11 | fill=T, stringsAsFactors = F) 12 | 13 | newentries = anti_join(olddiet, olddiet2, 14 | by = c("Common_Name", "Scientific_Name", "Subspecies", "Family", 15 | "Taxonomy", "Longitude_dd", "Latitude_dd", "Altitude_min_m", 16 | "Altitude_mean_m", "Altitude_max_m", "Location_Region", 17 | "Location_Specific", "Habitat_type", "Observation_Month_Begin", 18 | "Observation_Year_Begin", "Observation_Month_End", "Observation_Year_End", 19 | "Observation_Season", "Prey_Kingdom", "Prey_Phylum", "Prey_Class", 20 | "Prey_Order", "Prey_Suborder", "Prey_Family", "Prey_Genus", 21 | "Prey_Scientific_Name", "Unidentified", "Prey_Name_Status", 22 | "Prey_Stage", "Prey_Part", "Prey_Common_Name", "Fraction_Diet", 23 | "Diet_Type", "Item_Sample_Size", "Bird_Sample_Size", "Study_Type", 24 | "Notes", "Entered_By", "Source")) 25 | 26 | newnames = newentries %>% 27 | select(Common_Name, Scientific_Name, Family) %>% 28 | distinct() 29 | 30 | 31 | 32 | 33 | # Getting fraction diet summaries 34 | -------------------------------------------------------------------------------- /cleaning/fraction_sum_check_cleaning.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/cleaning/fraction_sum_check_cleaning.txt -------------------------------------------------------------------------------- /cleaning/martin1961_format_conversion.r: -------------------------------------------------------------------------------- 1 | # Convert data from Martin 1961 in the following format: 2 | 3 | #Common_Name Scientific_Name Observation_Season Fraction_Diet_Plant Bird_Sample_Size 4 | #Brant Branta bernicla Winter 1 60 5 | #Brant Branta bernicla Spring 0.93 33 6 | 7 | # into Diet Database format 8 | 9 | martin = read.table('AvianDietDatabase_Martin_1961.txt', header = T, sep = '\t', quote = '"', 10 | stringsAsFactors = FALSE) 11 | 12 | martindb = data.frame( 13 | Common_Name = rep(martin$Common_Name, each = 2), 14 | Scientific_Name = rep(martin$Scientific_Name, each = 2), 15 | Subspecies = NA, 16 | Family = NA, 17 | Taxonomy = NA, 18 | Longitude_dd = NA, 19 | Latitude_dd = NA, 20 | Altitude_min_m = NA, 21 | Altitude_mean_m = NA, 22 | Altitude_max_m = NA, 23 | Location_Region = rep('North America', 2*nrow(martin)), 24 | Location_Specific = NA, 25 | Habitat_type = NA, 26 | Observation_Month_Begin = NA, 27 | Observation_Year_Begin = NA, 28 | Observation_Month_End = NA, 29 | Observation_Year_End = rep(1961, 2*nrow(martin)), 30 | Observation_Season = rep(martin$Observation_Season, each = 2), 31 | Prey_Kingdom = rep(c('Plantae', 'Animalia'), nrow(martin)), 32 | Prey_Phylum = NA, 33 | Prey_Class = NA, 34 | Prey_Order = NA, 35 | Prey_Suborder = NA, 36 | Prey_Family = NA, 37 | Prey_Genus = NA, 38 | Prey_Scientific_Name = NA, 39 | Unidentified = rep('no', 2*nrow(martin)), 40 | Prey_Name_ITIS_ID = rep(c(202422,202423), nrow(martin)), 41 | Prey_Name_Status = rep('verified', 2*nrow(martin)), 42 | Prey_Stage = NA, 43 | Prey_Part = NA, 44 | Prey_Common_Name = NA, 45 | Fraction_Diet = as.vector(matrix(c(martin$Fraction_Diet_Plant, 1 - martin$Fraction_Diet_Plant), 46 | ncol = nrow(martin), byrow = TRUE)), 47 | Diet_Type = rep('Wt_or_Vol', 2*nrow(martin)), 48 | Item_Sample_Size = NA, 49 | Bird_Sample_Size = rep(martin$Bird_Sample_Size, each = 2), 50 | Sites = NA, 51 | Study_Type = NA, 52 | Notes = NA, 53 | Entered_By = rep('AHH', 2*nrow(martin)), 54 | Source = rep('Martin, A. C. 1961. American wildlife & plants : a guide to wildlife food habits : the use of trees, shrubs, weeds, and herbs by birds and mammals of the United States. Dover Publications, New York, 500 pp.', 2) 55 | ) 56 | 57 | martindb = martindb[martindb$Fraction_Diet != 0,] 58 | write.table(martindb, 'AvianDietDatabase_Martin_1961_cleaned.txt', sep = '\t', row.names = FALSE) 59 | -------------------------------------------------------------------------------- /cleaning/name_changes.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/cleaning/name_changes.txt -------------------------------------------------------------------------------- /cleaning/name_cleaning_instructions.md: -------------------------------------------------------------------------------- 1 | # Instructions for taxonomic name cleaning and filling in Prey_Name_ITIS_ID 2 | 3 | Taxonomic names may have changed since the study reporting them was originally 4 | published. Errors may also creep into taxonomic names via typos during data 5 | entry. 6 | 7 | The `clean_all_names()` function goes through each taxonomic level of prey 8 | names and finds the ITIS ID number of names that match, and flags the names that 9 | do not match. Here's an example using a test file with some known name problems. 10 | 11 | ``` 12 | > clean = clean_all_names('cleaning/test_namereplace_db.txt') 13 | 14 | [1] "1 out of 1" 15 | 16 | Retrieving data for taxon 'Stellaria' 17 | 18 | tsn target commonNames nameUsage 19 | 1 915378 Alsophila mostellaria NA accepted 20 | 2 44913 Cristellaria NA invalid 21 | 3 23670 Harrimanella stellariana NA not accepted 22 | 4 524470 Phlox bifida ssp. stellaria cleft phlox accepted 23 | 5 538671 Phlox bifida var. stellaria NA not accepted 24 | 6 518969 Phlox stellaria NA not accepted 25 | 7 20357 Pseudostellaria NA accepted 26 | 8 20358 Pseudostellaria jamesiana sticky-starwort,sticky starwort,tuber starwort accepted 27 | 9 823597 Pseudostellaria oxyphylla robust starwort accepted 28 | 10 823583 Pseudostellaria sierrae NA accepted 29 | 11 20163 Stellaria starwort accepted 30 | 12 989178 Stellaria NA invalid 31 | ... 32 | More than one TSN found for taxon 'Stellaria'! 33 | 34 | Enter rownumber of taxon (other inputs will return 'NA'): 35 | 36 | 1: 37 | ``` 38 | Name cleaning will be an interactive process, as the computer will frequently need 39 | your input to know how to proceed. In this case, the first name it tries to look up 40 | is 'Stellaria', and it has found multiple potential matches (of which I'm only showing 41 | the first 12). As it indicates, you must decide which of these entities is the one 42 | you want. Rows 11 and 12 are the only ones with just a simple genus name 'Stellaria', 43 | and Row 12 says that entity is invalid while the name in Row 11 is accepted. Thus we would 44 | type '11' and hit Enter, and R would move on to the next names to clean. 45 | 46 | ``` 47 | Input accepted, took taxon 'Stellaria'. 48 | 49 | [1] "1 out of 1" 50 | 51 | Retrieving data for taxon 'Acarina' 52 | 53 | [1] "1 out of 2" 54 | 55 | Retrieving data for taxon 'Rodentia' 56 | 57 | [1] "2 out of 2" 58 | 59 | Retrieving data for taxon 'Reptilia/Amphibia' 60 | 61 | [1] "1 out of 3" 62 | 63 | Retrieving data for taxon 'Streptophyta' 64 | 65 | [1] "2 out of 3" 66 | 67 | Retrieving data for taxon 'Foraminifera' 68 | 69 | tsn target commonNames nameUsage 70 | 1 879150 Edilemma foraminifera NA valid 71 | 2 1651 Paraphysomonas foraminifera NA accepted 72 | 73 | More than one TSN found for taxon 'Foraminifera'! 74 | 75 | Enter rownumber of taxon (other inputs will return 'NA'): 76 | 77 | 1: 78 | ``` 79 | The next several names R seems to know how to treat, until we get down to 80 | Foraminifera. In this case, it lists two individual species names, 81 | neither of which reflects the broad overall taxonomic group that are the 82 | [Foraminifera](https://en.wikipedia.org/wiki/Foraminifera). If there is 83 | no match (which will be the case with an old outdated name, as well), then 84 | simply hit enter. 85 | 86 | When the function has finished, we've created an object called clean, which has 87 | two elements, one called `cleandb` and one called `badnames`. Each of these objects 88 | gets saved to the same folder that the original file was read in from: 89 | "_clean.txt" and "_badnames.txt" 90 | ``` 91 | > names(clean) 92 | [1] "cleandb" "badnames" 93 | 94 | > clean$cleandb 95 | 96 | Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Unidentified Prey_Name_ITIS_ID 97 | 1 Plantae Tracheophyta Magnoliopsida Caryophyllales NA Caryophyllaceae Stellaria NA 20163 98 | 2 Plantae Tracheophyta Magnoliopsida Caryophyllales NA Caryophyllaceae Stellaria NA 20163 99 | 3 Plantae Streptophyta NA NA yes unverified 100 | 4 Plantae Streptophyta NA NA yes unverified 101 | 5 Plantae Streptophyta NA NA yes unverified 102 | 6 Foraminifera NA NA unverified 103 | 7 Foraminifera NA NA unverified 104 | 8 Bacillariophyta NA NA unverified 105 | 9 Bacillariophyta NA NA unverified 106 | 10 Animalia Chordata Rodentia NA NA no unverified 107 | 11 Animalia Chordata Reptilia/Amphibia NA NA yes unverified 108 | 12 Animalia Chordata Rodentia NA NA no unverified 109 | 13 Animalia Chordata Reptilia/Amphibia NA NA no unverified 110 | 14 Animalia Arthropoda Arachnida Acarina NA NA unverified 111 | 15 Animalia Arthropoda Arachnida Acarina NA NA unverified 112 | 16 Animalia Arthropoda Insecta Acarina NA NA unverified 113 | ``` 114 | The `cleandb` object is simply a version of the original database but with prey 115 | taxonomic name info updated when it was obvious how to do so. Note that in the first 116 | two rows corresponding to the Genus 'Stellaria', Prey_Phylum was changed from 117 | 'Streptophyta' to 'Tracheophyta' according to ITIS taxonomy, and the ITIS ID 118 | was added. 119 | 120 | The `badnames` object is a list of names that did not match the ITIS database 121 | at the taxonomic level specified: 122 | ``` 123 | > clean$badnames 124 | level name condition 125 | 1 Order Acarina wrong rank; too low 126 | 2 Class Rodentia wrong rank; too high 127 | 3 Class Reptilia/Amphibia unmatched 128 | 4 Phylum Streptophyta wrong rank; too low 129 | 5 Phylum Foraminifera unmatched 130 | 6 Phylum Bacillariophyta wrong rank; too low 131 | 7 Kingdom unaccepted name 132 | ``` 133 | These are names that you will have to decide how to fix or treat, and in some cases 134 | the 'condition' column can help. 135 | 136 | 137 | 1) To begin cleaning, first find and open the PDF of the study (given in the "study" column) this name 138 | appeared in. Most pdfs should be in the HurlbertLab folder > Databases > DietDatabase > 139 | Papers with data. If the paper is not in this folder, track it down online and 140 | save a PDF here. 141 | 142 | 2) Search the PDF for the unmatched name (use Ctrl-F). 143 | 144 | 3) If you find the unmatched name, pay attention to any contextual clues about 145 | what organism the name refers to. For example, the Beal (1912) source refers to a 146 | species called "Megilla maculata". This species is listed under Coleoptera. 147 | 148 | 6) Now paste this name into the [Global Names Resolver](http://resolver.globalnames.org/). 149 | If we are lucky, it will link to the currently accepted taxonomic name for that entity. 150 | If so, make sure the higher classification matches up with whatever info you gleaned 151 | from the original source. 152 | 153 | 7) If the name you pasted in does not generate any results in the Global Names Resolver, 154 | then try Google. Again, you are looking for clues for what this name refers to. In the 155 | case of "Megilla maculata", the first hit in Google is for a study called "Notes on the 156 | parasite of the Spotted Lady-Beetle (Megilla maculata)". Great! So, now try Googling 157 | "Spotted lady-beetle". This points us towards web entries for "Coleomegilla maculata", which 158 | certainly makes sense. Let's paste this new name in the [Global Names Resolver](http://resolver.globalnames.org/) 159 | just to be sure. Yes, looks good. 160 | 161 | 8) Now we want to provide a conversion table to tell R how to fix these names. Open the 162 | the 'badnames' file in the cleaning folder on your machine and open up this file in Excel. 163 | Add two new columns on to the right hand side, one called 'replacewith', and one called 'notes'. 164 | 165 | For problem names which were the result of a typo or taxnomic name update, you can 166 | put the corrected or updated name in the 'replacewith' column. 167 | 168 | If a taxonomic update also requires editing other fields as well, this can be done in the notes 169 | column. As long as the updated name is a valid ITIS name at the specificed taxonomic 170 | level, then usually no notes will be required. 171 | 172 | However, in the example above, 'Rodentia' is problematic because it is not a Class, 173 | but rather an Order within Class Mammalia. Thus, we would write 'Mammalia' in the 'replacewith' 174 | column and in the 'notes' column we would type 'Order = Rodentia'. 175 | 176 | In the case of 'Acarina', we find that not only is the name outdated (the currently 177 | accepted name is ['Acari'](https://www.itis.gov/servlet/SingleRpt/SingleRpt?search_topic=TSN&search_value=733321#null)), 178 | but it is not an Order, but rather a Subclass. Thus we would leave 'replacewith' blank 179 | because this taxonomic entity does not specify any particular order, and in the 'notes' 180 | field we might write 'Class = Arachnida & Phylum = Arthropoda & Kingdom = Animalia'. 181 | We can specify as many fields as we would like as long as each phrase of the form 182 | '(fieldname) = (value)' is separated by a '&'. 183 | 184 | 185 | 186 | As always, if you have any questions, don't hesitate to ask me! 187 | 188 | -------------------------------------------------------------------------------- /cleaning/name_cleaning_test_data.txt: -------------------------------------------------------------------------------- 1 | Common_Name Scientific_Name Subspecies Family Taxonomy Longitude_dd Latitude_dd Altitude_min_m Altitude_mean_m Altitude_max_m Location_Region Location_Specific Habitat_type Observation_Month_Begin Observation_Year_Begin Observation_Month_End Observation_Year_End Observation_Season Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Unidentified Prey_Name_ITIS_ID Prey_Name_Status Prey_Stage Prey_Part Prey_Common_Name Fraction_Diet Diet_Type Item_Sample_Size Bird_Sample_Size Sites Study_Type Notes Entered_By Source 2 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 2 1947 4 1947 Spring a 1 whatwhat Ericales NA Ericaceae NA NA yes NA seed Seeds 0.048 Wt_or_Vol NA 21 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 3 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 2 1947 4 1947 Spring b 2 whatwhat Ericales NA Ericaceae yes NA seed Seeds 0.048 Occurrence NA 21 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 4 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 5 1947 7 1947 Summer c 3 whatwhat Ericales NA Arctostaphylos uva-ursi yes NA flower Blossoms 0.027 Wt_or_Vol NA 197 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 5 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 5 1947 7 1947 Summer d 4 whatwhat Ericales NA NA NA NA yes NA seed Seeds 0.02 Wt_or_Vol NA 197 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 6 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 5 1947 7 1947 Summer e 5 whatwhat Ericales NA yes NA flower Blossoms 0.025 Occurrence NA 197 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 7 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 5 1947 7 1947 Summer f Tracheophyta NA yes NA seed Seeds 0.066 Occurrence NA 197 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 8 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 8 1947 10 1947 Fall g Tracheophyta NA NA NA NA NA NA yes NA seed Seeds 0.012 Wt_or_Vol NA 207 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 9 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 8 1947 10 1947 Fall Plantae Tracheophyta whatwhat uhuh NA noway Arctostaphylos NA yes NA seed Seeds 0.005 Occurrence NA 207 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 10 | Western Scrub-Jay Aphelocoma californica NA Corvidae "AOU 7th ed., 54th supplement" NA NA NA NA NA California NA agriculture 1 NA 12 NA NA Plantae NA no NA fruit manzanita 0.012 Occurrence NA 326 NA stomach contents "article has common name listed as California Jay, but current common name is Western Scrub-jay" RQ "Beal, F. E. L. 1910. Birds of California in relation to fruit industry, Pt. 2. U.S. Dept. Agricult. Biol. Surv. Bull. 34:1-96." 11 | California Quail Callipepla californica NA Odontophoridae "AOU 7th ed., 54th supplement" NA NA NA NA NA California NA agriculture 1 NA 12 NA NA Plantae NA NA NA NA NA NA NA no NA fruit manzanita 0.003 Occurrence NA 619 NA stomach contents "specimens were collected in every month except for May; study does not give year, although it can be assumed to be close to or before 1927" RQ "Beal, F. E. L. 1910. Birds of California in relation to fruit industry, Pt. 2. U.S. Dept. Agricult. Biol. Surv. Bull. 34:1-96." 12 | Acorn Woodpecker Melanerpes formicivorus NA Picdae "AOU 7th ed., 54th supplement" NA NA NA NA NA Oregon Rogue River Valley and Willamette Valley agriculture; wetland; woodland 11 1924 3 1926 All Gooberycop whatwhat Ericales NA NA NA NO NA fruit manzanitas 0.063463282 Wt_or_Vol NA 72 NA stomach contents "data originally included grit/mineral matter as percent of stomach volume, but it was taken out and only the total volume animal/plant were set as the measure of total Wt_or_Vol" JB "Neff, J. A. 1928. A study of the economic status of the common woodpeckers in relation to Oregon horticulture. Free Press Print, Marionville, MO." 13 | Spruce Grouse Falcipennis canadensis NA Phasianidae eBird Clements Checklist v2016 NA NA NA NA NA Montana Whitefish Range NA 9 1960 10 1962 NA Gooberycop Streptophyta whatwhat uhuh NA Ericaceae Arctostaphylos Arctostaphylos uva-ursi NA NA fruit NA 0.001 Wt_or_Vol 29 56 NA stomach contents NA AMO "Jonkel, C.J. & Greer, K.R. (1963) Fall food habits of spruce grouse in northwest Montana. Journal of Wildlife Management, 27, 593-596." 14 | test test test NA NA NA NA NA NA NA NA NA NA NA NA NA NA Animalia Arthropoda Insecta Coleoptera 15 | test test test NA NA NA NA NA NA NA NA NA NA NA NA NA NA Animalia Anthopoda Coleomegilla 16 | test test test NA NA NA NA NA NA NA NA NA NA NA NA NA NA Animalia Arthropoda Insecta Bombus 17 | test test test NA NA NA NA NA NA NA NA NA NA NA NA NA NA Insecta Drosophilidae 18 | test test test NA NA NA NA NA NA NA NA NA NA NA NA NA NA Insecta typo_typo 19 | test test test NA NA NA NA NA NA NA NA NA NA NA NA NA NA Insecta typo_typo 20 | name specified in the wrong rank should get saved to problem names NA NA NA NA NA Animalia Oligochaeta 21 | name specified in the wrong rank should get saved to problem names NA NA NA NA NA Animalia Oligochaeta 22 | name specified in the wrong rank should get saved to problem names NA NA NA NA NA wrong Oligochaeta 23 | -------------------------------------------------------------------------------- /cleaning/prey_parts.txt: -------------------------------------------------------------------------------- 1 | Prey_Part n replacewith other 2 | NA 11240 NA 3 | 10240 NA 4 | seed 2732 seed 5 | fruit 516 fruit 6 | leaf 303 vegetation 7 | vegetation 190 vegetation 8 | fiber 66 vegetation 9 | tuber 57 root 10 | root 45 root 11 | flower 39 flower 12 | berries 36 fruit 13 | shoots 33 vegetation 14 | carrion 29 NA 15 | epidermis 29 vegetation 16 | stem 26 vegetation 17 | bud 25 flower 18 | 22 NA 19 | rhizome 21 root 20 | feather 12 NA 21 | mast 12 seed 22 | plant material 12 NA 23 | propagule 11 seed 24 | shell 11 NA 25 | achene 10 seed 26 | stolon 10 vegetation 27 | caryopsis 9 seed 28 | needles 8 vegetation 29 | bulb 7 root 30 | nutlet 7 seed 31 | crushed animal matter 6 NA 32 | crushed plant matter 6 NA 33 | spore 6 spore 34 | fruit; seed 5 fruit 35 | mandible 5 NA 36 | pollen 5 pollen 37 | seed; fruit 5 fruit 38 | seedhead 5 seed 39 | acorn 4 seed 40 | anthers 4 pollen 41 | catkin 4 seed 42 | gall 4 gall 43 | oogonium 4 oogonium 44 | rootstalk 4 root 45 | subterranean parts 4 root 46 | bulbil 3 vegetation 47 | chitin 3 NA 48 | egg case 3 egg 49 | egg shell 3 egg 50 | nut 3 seed 51 | plant 3 NA 52 | tooth 3 NA 53 | acorn cotyledons 2 seed 54 | bark 2 bark 55 | bone; teeth 2 NA 56 | bud; flower 2 flower 57 | cocoon 2 NA Prey_Stage = pupa 58 | corm 2 root 59 | drupes 2 fruit 60 | dung 2 dung 61 | egg sacs 2 egg 62 | fibers 2 vegetation 63 | hair 2 NA 64 | kernel 2 seed 65 | lichens 2 NA Prey_Kingdom = Fungi & Prey_Phylum = Ascomycota & Prey_Common_Name = lichen 66 | pits; seed 2 seed 67 | puff ball 2 NA Prey_Kingdom = Fungi & Prey_Phylum = Basidiomycota 68 | rootstock 2 root 69 | scales 2 NA 70 | skin; pulp 2 fruit 71 | sporocarp 2 NA 72 | tubercle 2 vegetation 73 | acorn mast 1 seed 74 | berries; small fruit; seed 1 fruit 75 | bone 1 NA 76 | bones 1 NA 77 | bud; leaf 1 vegetation 78 | bud; shoots 1 vegetation 79 | cambium 1 vegetation 80 | capsule 1 spore 81 | carcass; afterbirth 1 NA 82 | cotyledon 1 seed 83 | down 1 NA 84 | elytra 1 NA 85 | feather; bone 1 NA 86 | feathers 1 NA 87 | fiber; pollen 1 pollen 88 | foliage 1 vegetation 89 | fragments 1 seed 90 | galls 1 gall 91 | jawbone 1 NA 92 | leg 1 NA 93 | oospore 1 spore 94 | petals 1 flower 95 | pulp 1 seed 96 | scute 1 NA 97 | seed; fruit (non-conifer) 1 seed 98 | seed; fruit pulp; catkin; etc 1 seed 99 | seedpod 1 seed 100 | skin 1 fruit 101 | skin; fruit 1 fruit 102 | sporangium 1 spore 103 | spore case 1 spore 104 | stamen 1 pollen 105 | submerged vegetation 1 vegetation 106 | tree sap 1 sap 107 | whole grain + fragments 1 seed 108 | -------------------------------------------------------------------------------- /cleaning/prey_stages.txt: -------------------------------------------------------------------------------- 1 | Prey_Stage n replacewith notes 2 | NA 13296 NA 3 | 11325 NA 4 | larvae 509 larva 5 | adult 481 adult 6 | juvenile 47 juvenile 7 | larvae; adult 42 NA 8 | pupae 40 pupa 9 | fledgling 36 juvenile 10 | egg 34 egg 11 | nymphs 28 larva 12 | larvae; pupae 16 CHECK 13 | adult; larvae 10 CHECK 14 | adult; egg 6 CHECK 15 | nymph 6 larva 16 | teneral 6 teneral 17 | eggs 4 egg 18 | larval 4 larva 19 | nestling 4 juvenile 20 | young 4 juvenile 21 | fawn 3 juvenile 22 | adult; cocoon 2 CHECK 23 | ducklings 2 juvenile 24 | egg; larvae; pupae 2 CHECK 25 | ephippia 2 egg 26 | larvae; adult; pupae 2 CHECK 27 | larvae; pupae; adult 2 CHECK 28 | naiads 2 larva 29 | pupa 2 pupa 30 | adult; larvae; pupae 1 CHECK 31 | adult; pupae 1 CHECK 32 | immature 1 juvenile 33 | parr 1 juvenile 34 | pupae; adult 1 pupa; adult 35 | seed 1 NA Prey_Part = seed 36 | smolts 1 juvenile 37 | statoblasts 1 statoblast 38 | tadpoles 1 larva 39 | -------------------------------------------------------------------------------- /cleaning/problem_names.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/cleaning/problem_names.txt -------------------------------------------------------------------------------- /cleaning/qa_qc_test_db.txt: -------------------------------------------------------------------------------- 1 | Common_Name Scientific_Name Subspecies Family Taxonomy Longitude_dd Latitude_dd Altitude_min_m Altitude_mean_m Altitude_max_m Location_Region Location_Specific Habitat_type Observation_Month_Begin Observation_Year_Begin Observation_Month_End Observation_Year_End Observation_Season Analysis_Number Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Inclusive_Prey_Taxon Prey_Name_ITIS_ID Prey_Name_Status Prey_Stage Prey_Part Prey_Common_Name Fraction_Diet Diet_Type Item_Sample_Size Bird_Sample_Size Sites Study_Type Notes Entered_By Source 2 | Eastern Towhee Pipilo erythrophthalmus NA Emberizidae eBird Clements Checklist v2016 -71.5745006 41.19177 NA NA NA Rode Island Multiple shrubland August 1993 10 1995 summer 1 Plantae verified fruit 1 Occurrence NA 21 12 Fecal contents MB test 3 | Eastern Towhee Pipilo erythrophthalmus NA Emberizidae eBird Clements Checklist v2016 -71.5745006 41.19177 NA NA NA Rhode Island Multiple scrubland 8 1993 10 1995 summer 1 Animalia Arthropoda Insecta 99208 verified NA 0.741 Occurrence NA 21 12 fecal contents MB test 4 | Eastern towhee Pipilo erythrophthalmus NA Emberizidae eBird Clements Checklist v2016 -715.745006 41.19177 NA NA NA Rhode Island Multiple shrubland 8 1993 10 1995 summer 1 Plantae verified fruit 0.814 Wt_or_Vol NA 21 12 fecal contents MB test 5 | Eastern Towhee Pipilo erythrophthalmus NA Emberizidae eBird Clements Checklist v2016 -71.5745006 41.19177 NA NA NA Rhode Island Multiple shrubland 8 1993 10 1995 summer 1 Animalia Arthropoda Insecta 99208 verified juvie NA 0.186 Wt_or_Vol NA 21 12 fecal contents MB test 6 | Swamp Sparrow Melospiza georgiana NA Emberizidae eBird Clements Checklist v2016 -71.5745006 41.19177 NA NA NA Rhode Island Multipe shrubland 8 1993 10 1995 summer 1 Plantae verified fruit 0.818 Occurrence NA 11 12 fecal contents MB test 7 | Swamp Sparrow Melospiza georgiana NA Emberizidae eBird Clements Checklist v2016 -71.5745006 41.19177 NA NA NA Rhode Island Multiple shrubland 8 1993 10 1995 summer 1 Animalia Arthropoda Insecta 99208 verified NA 0.727 Occurrence NA 11 12 fecal contents MB test 8 | Swamp Sparrow Melospiza georgiana NA Emberizidae eBird Clements Checklist v2016 -71.5745006 41.19177 NA NA NA Rhode Island Multiple shrubland 8 1993 10 1995 summer 1 Plantae verified fruity 0.518 Wt_or_Vol NA 11 12 fecal contents MB test 9 | Swamp Sparrow Melospiza georgina NA Emberizidae eBird Clements Checklist v2016 -71.5745006 41.19177 NA NA NA Rhode Island Multiple shrubland 8 1993 10 1995 summer 1 Animalia Arthropoda Insecta 99208 verified larva;adult NA 0.482 Wt_or_vol NA 11 12 fecal contents MB test 10 | White-throated Sparrow Zonotrichia albicollis NA Emberizidae eBird Clements Checklist v2016 -71.5745006 41.19177 NA NA NA Rhode Island Multiple shrubland 8 1993 10 1995 fall 1 Plantae verified fruit 92.1 Wt_or_Vol NA 38 12 fecal contents MB test 11 | White-throated Sparrow Zonotrichia albicollis NA Emberizidae eBird Clements Checklist v2015 -71.5745006 41.19177 NA NA NA Rhode Island Multiple shrubland 8 1993 10 1995 fall 1 Animalia Arthropoda Insecta 99208 verified adult;larva NA 84.2 Wt_or_Vol NA 38 12 fecal contents MB test 12 | White-throated Sparrow Zonotrichia albicollis NA Emberizidae eBird Clements Checklist v2016 -71.5745006 41.19177 NA NA NA Rhode Island Multiple shrubland 8 1993 10 1995 summer 1 Plantae verified fruit 0.621 Wt_or_Vol NA 38 12 fecal contents MB test 13 | White-throated Sparrow Zonotrichia albicollis NA Emberizidae eBird Clements Checklist v2016 -71.5745006 41.19177 NA NA NA Rhode Island Multiple shrubland 8 1993 10 1995 summer 1 Animalia Arthropoda Insecta 99208 accepted NA 0.4 Wt_or_Vol NA 38 12 fecal contents MB test 14 | -------------------------------------------------------------------------------- /cleaning/test_namereplace_db.txt: -------------------------------------------------------------------------------- 1 | Common_Name Scientific_Name Subspecies Family Taxonomy Longitude_dd Latitude_dd Altitude_min_m Altitude_mean_m Altitude_max_m Location_Region Location_Specific Habitat_type Observation_Month_Begin Observation_Year_Begin Observation_Month_End Observation_Year_End Observation_Season Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Unidentified Prey_Name_Status Prey_Stage Prey_Part Prey_Common_Name Fraction_Diet Diet_Type Item_Sample_Size Bird_Sample_Size Sites Study_Type Notes Entered_By Source 2 | House Sparrow Passer domesticus NA Passeridae "AOU 7th ed., 54th supplement" NA NA NA NA NA Pennsylvania Centre County agriculture 4 1981 7 1981 Summer Plantae Streptophyta Magnoliopsida Caryophyllales Caryophyllaceae Stellaria 0.168 Occurrence NA 267 9 stomach contents; emetic; fecal examination "Gavett, A. P. and J. S. Wakeley. 1986. Diets of House Sparrows in urban and rural habitats. Wilson Bull. 98:137-144." 3 | House Sparrow Passer domesticus NA Passeridae "AOU 7th ed., 54th supplement" NA NA NA NA NA Pennsylvania Centre County urban 4 1981 7 1981 Summer Plantae Streptophyta Magnoliopsida Caryophyllales Caryophyllaceae Stellaria 0.106 Occurrence NA 218 15 stomach contents; emetic; fecal examination "Gavett, A. P. and J. S. Wakeley. 1986. Diets of House Sparrows in urban and rural habitats. Wilson Bull. 98:137-144." 4 | House Sparrow Passer domesticus NA Passeridae "AOU 7th ed., 54th supplement" NA NA NA NA NA Pennsylvania Centre County agriculture 4 1981 7 1981 Summer Plantae Streptophyta yes 0.037 Occurrence NA 267 9 stomach contents; emetic; fecal examination "Gavett, A. P. and J. S. Wakeley. 1986. Diets of House Sparrows in urban and rural habitats. Wilson Bull. 98:137-144." 5 | House Sparrow Passer domesticus NA Passeridae "AOU 7th ed., 54th supplement" NA NA NA NA NA Pennsylvania Centre County urban 4 1981 7 1981 Summer Plantae Streptophyta yes 0.096 Occurrence NA 218 15 stomach contents; emetic; fecal examination "Gavett, A. P. and J. S. Wakeley. 1986. Diets of House Sparrows in urban and rural habitats. Wilson Bull. 98:137-144." 6 | House Sparrow Passer domesticus NA Passeridae "AOU 7th ed., 54th supplement" NA NA NA NA NA Pennsylvania Centre County agriculture 4 1981 7 1981 Summer Plantae Streptophyta yes 0.217 Occurrence NA 267 9 stomach contents; emetic; fecal examination "Gavett, A. P. and J. S. Wakeley. 1986. Diets of House Sparrows in urban and rural habitats. Wilson Bull. 98:137-144." 7 | Northern Shoveler Anas clypeata NA Anatidae eBird Clements Checklist v2016 NA NA NA NA NA Texas NA wetland 10 1982 5 1984 NA NA Foraminifera NA NA NA NA NA NA NA NA NA NA foraminifera 0.1633 Wt_or_Vol 20 134 NA stomach contents NA AMO "Tietje, W.D. & Teer, J.G. (1996) Winter feeding ecology of northern shovelers on freshwater and saline wetlands in south Texas. Journal of Wildlife Management, 60, 843-855." 8 | Steller's Eider Polysticta stelleri NA Anatidae eBird Clements Checklist v2016 NA NA NA NA NA Alaska NA NA NA 1914 NA 1914 NA NA Foraminifera NA NA NA NA NA NA NA NA NA NA foraminifera 0.0107 Wt_or_Vol 19 66 NA stomach contents NA AMO "Cottam, C. (1939) Food habits of North American diving ducks. United States Department of Agriculture Technical Bulletin, 643, 1-140." 9 | Common Eider Somateria mollissima NA Anatidae eBird Clements Checklist v2016 NA NA NA NA NA Canada St. Lawrence estuary NA 6 1972 9 1972 NA NA Bacillariophyta NA NA NA NA NA NA NA NA NA NA diatoms 0.005 Occurrence 24 162 NA stomach contents Data collected from male and female individuals AMO "Cantin, M., Bedard, J. & Milne, H. (1974) The food and feeding of common eiders in the St. Lawrence estuary in summer. Canadian Journal of Zoology, 52, 319-334." 10 | Common Shelduck Tadorna tadorna NA Anatidae eBird Clements Checklist v2016 NA NA NA NA NA Netherlands Ooesterschelde NA 11 1990 12 1990 NA NA Bacillariophyta NA NA NA NA NA NA NA NA NA NA diatoms 0.1 Occurrence 4 NA NA fecal examination NA AMO "Meininger, P.L. & Snoek, H. (1992) Non-breeding Shelduck Tadorna tadorna in the southwest Netherlands: effects of habitat changes on distribution, numbers, moulting sites and food. Wildfowl, 43, 139-151." 11 | Bald Eagle Haliaeetus leucocephalus NA Accipitridae "AOU 7th ed., 54th supplement" NA NA NA NA NA Oregon Williamette Valley grassland; deciduous forest 12 1986 3 1987 Winter Animalia Chordata Rodentia no verified small rodent 0.22 Items 45 20 1 fecal examination the site habitat hosts many domestic sheep; maximum number of birds was 20; 68.2% of birds observed were subadult CS "Dellasala, D. A., C. L. Thomas, and R. G. Anthony. 1989. Use of domestic sheep carrion by Bald Eagles wintering in the Willamette Valley, Oregon. Northwest Sci. 63:104-108." 12 | Bald Eagle Haliaeetus leucocephalus NA Accipitridae "AOU 7th ed., 54th supplement" NA NA 500 NA 1500 Arizona Verde and Salt rivers shrubland NA 1983 NA 1985 NA Animalia Chordata Reptilia/Amphibia yes verified reptile/amphibian 0.00476 Items 1471 NA 14 behavioral observation 14 nests were observed; CS "Grubb, T. G. 1995. Food habits of Bald Eagles breeding in the Arizona desert. Wilson Bull. 107:258-274." 13 | Bald Eagle Haliaeetus leucocephalus NA Accipitridae "AOU 7th ed., 54th supplement" NA NA NA NA NA Alaska Robert Islands NA 6 1971 8 1971 NA Animalia Chordata Rodentia no verified Rodent 0.0259 Items 116 6 1 behavioral observation "study area is beach; 3 pairs of eagles (at 3 different nests) were studied; I combined results from nests 2, 5, and 10 to get the aggregate fractions per prey species" CS "Ofelt, C. H. 1975. Food habits of nesting Bald Eagles in southeast Alaska. Condor 77:337-338." 14 | Broad-winged Hawk Buteo platypterus NA Accipitridae "AOU 7th ed., 57th supplement" NA NA NA NA NA Missouri Multiple multiple 9 1978 7 1985 NA Animalia Chordata Reptilia/Amphibia no NA NA reptiles and amphibians 0.191489362 Items 47 NA NA observation Study based on observation of prey capture PW "Toland, B. 1986. Hunting success of some Missouri raptors. Wilson Bull. 98:116-125." 15 | Ovenbird Seiurus aurocapilla NA Parulidae "AOU 7th ed., 52nd supplement" NA NA 5 NA 630 Jamaica shrubland; forest 11 1993 3 1997 Winter Animalia Arthropoda Arachnida Acarina verified 0.06 Occurrence 2137 53 4 emetic "When seeds were excluded from the analysis, Ovenbird diets were similar across habitats" "Strong, A. M. 2000. Divergent foraging strategies of two neotropical migrant warblers: Implications for winter habitat use. Auk 117(2):381-392." 16 | Winter Wren Troglodytes hiemalis NA Troglodytidae "AOU 7th ed., 54th supplement" NA NA NA NA NA British Columbia NA NA NA NA Animalia Arthropoda Arachnida Acarina verified 0.22 Occurrence NA 122 NA emetic Study data incomplete - waiting for complete dissertation to arrive via interlibrary loan "Mclachlin, R. A. 1983. Dispersion of the Western Winter Wren (Troglodytes troglodytes) in the coastal western hemlock forest at the University of British Columbia Research Forest in southwestern British Columbia. Phd Thesis. Univ. of British Columbia, Vancouver." 17 | Eastern Phoebe Sayornis phoebe NA Tyrannidae "AOU 7th ed., 54th supplement" NA NA NA NA NA Multiple Multiple 3 1911 10 1911 Summer Animalia Arthropoda Insecta Acarina verified 0.005 Occurrence NA 370 NA stomach contents "Broad study in 19 states, distric of columbia and Canada" "Beal, F. E. L. 1912. Food of our more important flycatchers. U.S. Dep. Agric. Biol. Surv. Bull. No. 44." 18 | -------------------------------------------------------------------------------- /cleaning/test_namereplace_db_badnames.txt: -------------------------------------------------------------------------------- 1 | "level" "name" "condition" 2 | "Order" "Acarina" "unmatched" 3 | "Class" "Rodentia" "wrong rank; too high" 4 | "Class" "Reptilia/Amphibia" "unmatched" 5 | "Phylum" "Streptophyta" "wrong rank; too low" 6 | "Phylum" "Foraminifera" "unmatched" 7 | "Phylum" "Bacillariophyta" "unmatched" 8 | "Kingdom" NA "unaccepted name" 9 | -------------------------------------------------------------------------------- /cleaning/test_namereplace_db_clean.txt: -------------------------------------------------------------------------------- 1 | "Common_Name" "Scientific_Name" "Subspecies" "Family" "Taxonomy" "Longitude_dd" "Latitude_dd" "Altitude_min_m" "Altitude_mean_m" "Altitude_max_m" "Location_Region" "Location_Specific" "Habitat_type" "Observation_Month_Begin" "Observation_Year_Begin" "Observation_Month_End" "Observation_Year_End" "Observation_Season" "Prey_Kingdom" "Prey_Phylum" "Prey_Class" "Prey_Order" "Prey_Suborder" "Prey_Family" "Prey_Genus" "Prey_Scientific_Name" "Unidentified" "Prey_Name_Status" "Prey_Stage" "Prey_Part" "Prey_Common_Name" "Fraction_Diet" "Diet_Type" "Item_Sample_Size" "Bird_Sample_Size" "Sites" "Study_Type" "Notes" "Entered_By" "Source" "Prey_Name_ITIS_ID" 2 | "House Sparrow" "Passer domesticus" NA "Passeridae" "AOU 7th ed., 54th supplement" NA NA NA NA NA "Pennsylvania" "Centre County" "agriculture" 4 1981 7 1981 "Summer" "Plantae" "Plantae" "Tracheophyta" "Magnoliopsida" "Caryophyllales" "Caryophyllaceae" "Caryophyllaceae" NA "" "verified" NA NA "" 0.168 "Occurrence" NA 267 9 "stomach contents; emetic; fecal examination" "" "" "Gavett, A. P. and J. S. Wakeley. 1986. Diets of House Sparrows in urban and rural habitats. Wilson Bull. 98:137-144." "20163" 3 | "House Sparrow" "Passer domesticus" NA "Passeridae" "AOU 7th ed., 54th supplement" NA NA NA NA NA "Pennsylvania" "Centre County" "urban" 4 1981 7 1981 "Summer" "Plantae" "Plantae" "Tracheophyta" "Magnoliopsida" "Caryophyllales" "Caryophyllaceae" "Caryophyllaceae" NA "" "verified" NA NA "" 0.106 "Occurrence" NA 218 15 "stomach contents; emetic; fecal examination" "" "" "Gavett, A. P. and J. S. Wakeley. 1986. Diets of House Sparrows in urban and rural habitats. Wilson Bull. 98:137-144." "20163" 4 | "House Sparrow" "Passer domesticus" NA "Passeridae" "AOU 7th ed., 54th supplement" NA NA NA NA NA "Pennsylvania" "Centre County" "agriculture" 4 1981 7 1981 "Summer" "Plantae" "Streptophyta" "" "" NA "" "" NA "yes" "" NA NA "" 0.037 "Occurrence" NA 267 9 "stomach contents; emetic; fecal examination" "" "" "Gavett, A. P. and J. S. Wakeley. 1986. Diets of House Sparrows in urban and rural habitats. Wilson Bull. 98:137-144." "20163" 5 | "House Sparrow" "Passer domesticus" NA "Passeridae" "AOU 7th ed., 54th supplement" NA NA NA NA NA "Pennsylvania" "Centre County" "urban" 4 1981 7 1981 "Summer" "Plantae" "Streptophyta" "" "" NA "" "" NA "yes" "" NA NA "" 0.096 "Occurrence" NA 218 15 "stomach contents; emetic; fecal examination" "" "" "Gavett, A. P. and J. S. Wakeley. 1986. Diets of House Sparrows in urban and rural habitats. Wilson Bull. 98:137-144." "20163" 6 | "House Sparrow" "Passer domesticus" NA "Passeridae" "AOU 7th ed., 54th supplement" NA NA NA NA NA "Pennsylvania" "Centre County" "agriculture" 4 1981 7 1981 "Summer" "Plantae" "Streptophyta" "" "" NA "" "" NA "yes" "" NA NA "" 0.217 "Occurrence" NA 267 9 "stomach contents; emetic; fecal examination" "" "" "Gavett, A. P. and J. S. Wakeley. 1986. Diets of House Sparrows in urban and rural habitats. Wilson Bull. 98:137-144." "20163" 7 | "Northern Shoveler" "Anas clypeata" NA "Anatidae" "eBird Clements Checklist v2016" NA NA NA NA NA "Texas" NA "wetland" 10 1982 5 1984 NA NA "Foraminifera" NA NA NA NA NA NA NA "unverified" NA NA "foraminifera" 0.1633 "Wt_or_Vol" 20 134 NA "stomach contents" NA "AMO" "Tietje, W.D. & Teer, J.G. (1996) Winter feeding ecology of northern shovelers on freshwater and saline wetlands in south Texas. Journal of Wildlife Management, 60, 843-855." "20163" 8 | "Steller's Eider" "Polysticta stelleri" NA "Anatidae" "eBird Clements Checklist v2016" NA NA NA NA NA "Alaska" NA NA NA 1914 NA 1914 NA NA "Foraminifera" NA NA NA NA NA NA NA "unverified" NA NA "foraminifera" 0.0107 "Wt_or_Vol" 19 66 NA "stomach contents" NA "AMO" "Cottam, C. (1939) Food habits of North American diving ducks. United States Department of Agriculture Technical Bulletin, 643, 1-140." "20163" 9 | "Common Eider" "Somateria mollissima" NA "Anatidae" "eBird Clements Checklist v2016" NA NA NA NA NA "Canada" "St. Lawrence estuary" NA 6 1972 9 1972 NA NA "Bacillariophyta" NA NA NA NA NA NA NA "unverified" NA NA "diatoms" 0.005 "Occurrence" 24 162 NA "stomach contents" "Data collected from male and female individuals" "AMO" "Cantin, M., Bedard, J. & Milne, H. (1974) The food and feeding of common eiders in the St. Lawrence estuary in summer. Canadian Journal of Zoology, 52, 319-334." "20163" 10 | "Common Shelduck" "Tadorna tadorna" NA "Anatidae" "eBird Clements Checklist v2016" NA NA NA NA NA "Netherlands" "Ooesterschelde" NA 11 1990 12 1990 NA NA "Bacillariophyta" NA NA NA NA NA NA NA "unverified" NA NA "diatoms" 0.1 "Occurrence" 4 NA NA "fecal examination" NA "AMO" "Meininger, P.L. & Snoek, H. (1992) Non-breeding Shelduck Tadorna tadorna in the southwest Netherlands: effects of habitat changes on distribution, numbers, moulting sites and food. Wildfowl, 43, 139-151." "20163" 11 | "Bald Eagle" "Haliaeetus leucocephalus" NA "Accipitridae" "AOU 7th ed., 54th supplement" NA NA NA NA NA "Oregon" "Williamette Valley" "grassland; deciduous forest" 12 1986 3 1987 "Winter" "Animalia" "Chordata" "Rodentia" "" NA "" "" NA "no" "verified" NA NA "small rodent" 0.22 "Items" 45 20 1 "fecal examination" "the site habitat hosts many domestic sheep; maximum number of birds was 20; 68.2% of birds observed were subadult" "CS" "Dellasala, D. A., C. L. Thomas, and R. G. Anthony. 1989. Use of domestic sheep carrion by Bald Eagles wintering in the Willamette Valley, Oregon. Northwest Sci. 63:104-108." "20163" 12 | "Bald Eagle" "Haliaeetus leucocephalus" NA "Accipitridae" "AOU 7th ed., 54th supplement" NA NA 500 NA 1500 "Arizona" "Verde and Salt rivers" "shrubland" NA 1983 NA 1985 NA "Animalia" "Chordata" "Reptilia/Amphibia" "" NA "" "" NA "yes" "verified" NA NA "reptile/amphibian" 0.00476 "Items" 1471 NA 14 "behavioral observation" "14 nests were observed; " "CS" "Grubb, T. G. 1995. Food habits of Bald Eagles breeding in the Arizona desert. Wilson Bull. 107:258-274." "20163" 13 | "Bald Eagle" "Haliaeetus leucocephalus" NA "Accipitridae" "AOU 7th ed., 54th supplement" NA NA NA NA NA "Alaska" "Robert Islands" NA 6 1971 8 1971 NA "Animalia" "Chordata" "Rodentia" "" NA "" "" NA "no" "verified" NA NA "Rodent" 0.0259 "Items" 116 6 1 "behavioral observation" "study area is beach; 3 pairs of eagles (at 3 different nests) were studied; I combined results from nests 2, 5, and 10 to get the aggregate fractions per prey species" "CS" "Ofelt, C. H. 1975. Food habits of nesting Bald Eagles in southeast Alaska. Condor 77:337-338." "20163" 14 | "Broad-winged Hawk" "Buteo platypterus" NA "Accipitridae" "AOU 7th ed., 57th supplement" NA NA NA NA NA "Missouri" "Multiple" "multiple" 9 1978 7 1985 NA "Animalia" "Chordata" "Reptilia/Amphibia" "" NA "" "" NA "no" "" NA NA "reptiles and amphibians" 0.191489362 "Items" 47 NA NA "observation" "Study based on observation of prey capture" "PW" "Toland, B. 1986. Hunting success of some Missouri raptors. Wilson Bull. 98:116-125." "20163" 15 | "Ovenbird" "Seiurus aurocapilla" NA "Parulidae" "AOU 7th ed., 52nd supplement" NA NA 5 NA 630 "Jamaica" "" "shrubland; forest" 11 1993 3 1997 "Winter" "Animalia" "Arthropoda" "Arachnida" "Acarina" NA "" "" NA "" "unverified" NA NA "" 0.06 "Occurrence" 2137 53 4 "emetic" "When seeds were excluded from the analysis, Ovenbird diets were similar across habitats" "" "Strong, A. M. 2000. Divergent foraging strategies of two neotropical migrant warblers: Implications for winter habitat use. Auk 117(2):381-392." "20163" 16 | "Winter Wren" "Troglodytes hiemalis" NA "Troglodytidae" "AOU 7th ed., 54th supplement" NA NA NA NA NA "British Columbia" "" "" NA NA NA NA "" "Animalia" "Arthropoda" "Arachnida" "Acarina" NA "" "" NA "" "unverified" NA NA "" 0.22 "Occurrence" NA 122 NA "emetic" "Study data incomplete - waiting for complete dissertation to arrive via interlibrary loan" "" "Mclachlin, R. A. 1983. Dispersion of the Western Winter Wren (Troglodytes troglodytes) in the coastal western hemlock forest at the University of British Columbia Research Forest in southwestern British Columbia. Phd Thesis. Univ. of British Columbia, Vancouver." "20163" 17 | "Eastern Phoebe" "Sayornis phoebe" NA "Tyrannidae" "AOU 7th ed., 54th supplement" NA NA NA NA NA "Multiple" "Multiple" "" 3 1911 10 1911 "Summer" "Animalia" "Arthropoda" "Insecta" "Acarina" NA "" "" NA "" "unverified" NA NA "" 0.005 "Occurrence" NA 370 NA "stomach contents" "Broad study in 19 states, distric of columbia and Canada" "" "Beal, F. E. L. 1912. Food of our more important flycatchers. U.S. Dep. Agric. Biol. Surv. Bull. No. 44." "20163" 18 | -------------------------------------------------------------------------------- /cleaning/test_probnames.txt: -------------------------------------------------------------------------------- 1 | level name condition replacewith notes 2 | Phylum Streptophyta wrong rank; too low Tracheophyta 3 | Phylum Foraminifera unmatched Retaria 4 | Phylum Bacillariophyta wrong rank; too low Ochrophyta Class = Bacillariophyceae 5 | Class Rodentia wrong rank; too high Mammalia Order = Rodentia 6 | Class Reptilia/Amphibia unmatched keep as is 7 | Order Acarina wrong rank; too low Class = Arachnida & Common_Name = mite 8 | Order Lepidoptera unmatched Lepidoptera 9 | -------------------------------------------------------------------------------- /contributors.md: -------------------------------------------------------------------------------- 1 | Contributors to the Avian Diet Database 2 | ======================================= 3 | 4 | * Michael Auriemma, 2014-2015 (Turdidae, Tyrranidae) 5 | * Jesse Barnes, 2015-2016 (Picidae, Sittidae) 6 | * Megan Blawas, 2017 (misc; error checking) 7 | * Cassandra Bridges, 2019 (misc) 8 | * Jericho Cook, 2018 (Vireonidae) 9 | * Rebecca Cooper, 2019 (Passerellidae) 10 | * Allison Duprey, 2018 (Barred Owl) 11 | * Josh Fox, 2020- (misc; error checking) 12 | * Gianna Glass, 2019 (Strigidae, misc) 13 | * Thomas Hennessey, 2016-2017 (Icteridae, Paridae, misc) 14 | * Allen Hurlbert, 2012-present 15 | * Jeremy Knowlton, 2012-2013 (Parulidae, Vireonidae) 16 | * Ellie Kremer, 2018-2019 (Rallidae, Phasianidae) 17 | * William Larsen, 2015 (Strigidae, Accipitridae, Tytonidae) 18 | * Emi McGeady, 2019-2020 (Laridae, Diomedeidae, Phaethontidae, misc) 19 | * Tara McKinnon, 2015-2016 (Accipitridae, Falconidae, Strigidae) 20 | * George Ni, 2019-2020 (Columbidae, Parulidae, misc) 21 | * Aaron Olsen, 2016 (Anatidae, Anhimidae, Anseranatidae, Cracidae, Megapodiidae, Numididae, Odonotophoridae) 22 | * Lexie Peeler, 2015 (Corvidae) 23 | * Rachel Quindlen, 2015-2016 (Cardinalidae, Hirundinidae, Parulidae, Tyrannidae, Vireonidae, Beal compendia) 24 | * Annabel Richards, 2017-2018 (Odontophoridae, Phasianidae) 25 | * Dong Ryu, 2017-2018 (Falconidae, misc) 26 | * Miguel Sandoval, 2014-2015 (Passerellidae) 27 | * Melissa Sawyer, 2019-2021 (Phalacrocoracidae, Podicipedidae, Rallidae, Phasianidae, Phaethontidae, Ardeidae, Threshkiornithidae, misc) 28 | * Caitlin Seyfried, 2015 (Accipitridae) 29 | * Savannah Swinea, 2016 (Columbidae, Apodidae, Ciconiidae, Icteridae, Cuculidae) 30 | * Jayne Willard, 2021 (Fregatidae) 31 | * Patrick Winner, 2016-2017 (Accipitridae, Alaudidae, Bombycillidae, Calcariidae, Certhiidae, Corvidae, Cuculidae, Emberizidae, Fringillidae, Icteridae, Laniidae, Mimidae, Motacillidae, Pandionidae, Picidae, Polioptilidae, Rallidae, Regulidae, Sittidae, Strigidae, Sturnidae, Troglodytidae, Tytonidae, Vireonidae, misc; error checking) 32 | * Andrew Zachman, 2018 (Phasianidae) 33 | -------------------------------------------------------------------------------- /data_priorities.md: -------------------------------------------------------------------------------- 1 | Avian Diet Database - Priorities for Data Entry 2 | =============================================== 3 | 4 | The following Orders are being targeted for our initial data publication. Please focus on 5 | finishing up everything within Passeriformes before working on other Orders. 6 | 7 | Note that Families for which I know that data remain to be entered are highlighted with parenthetical notes. However, it should probably be confirmed that no References remain to be checked for the other Families. 8 | 9 | ### Accipitriformes 10 | --Accipitridae 11 | --Cathartidae 12 | 13 | ### Ciconiiformes 14 | --Ciconiidae 15 | 16 | ### Columbiformes 17 | --Columbidae 18 | 19 | ### Cuculiformes 20 | --Cuculidae 21 | 22 | ### Passeriformes 23 | --Aegithalidae 24 | --Alaudidae 25 | --Bombycillidae 26 | --Calcariidae 27 | --Cardinalidae 28 | --Certhiidae 29 | --Cinclidae 30 | --Corvidae 31 | --Emberizide 32 | --Fringillidae 33 | --Hirundinidae 34 | --Icteridae 35 | --Laniidae 36 | --Mimidae 37 | --Motacillidae 38 | --Paridae 39 | --Parulidae 40 | --Passeridae 41 | --Polioptilidae 42 | --Ptilogonatidae 43 | --Regulidae 44 | --Remizidae 45 | --Sittidae 46 | --Sturnidae 47 | --Sylviidae 48 | --Timaliidae 49 | --Troglodytidae 50 | --Turdidae 51 | --Tyrannidae 52 | --Vireonidae 53 | 54 | ### Piciformes 55 | --Picidae 56 | 57 | ### Psittaciformes 58 | --Psittacidae 59 | 60 | ### Strigiformes 61 | --Strigidae 62 | --Tytonidae 63 | 64 | ### Trogoniformes 65 | --Trogonidae 66 | 67 | ### Caprimulgiformes 68 | --Apodidae 69 | --Caprimulgidae 70 | --Trochilidae 71 | 72 | ### Falconiformes 73 | --Falconidae 74 | 75 | ### Coraciiformes 76 | --Alcedinidae 77 | 78 | ### Gruiformes 79 | --Aramidae 80 | --Gruidae 81 | --Rallidae 82 | 83 | ### Galliformes 84 | --Odontophoridae 85 | --Cracidae 86 | --Phasianidae 87 | 88 | ### Suliformes 89 | --Phalacrocoracidae 90 | 91 | ### Pelecaniformes 92 | --Pelecanidae 93 | --Threskiornithidae 94 | --Ardeidae 95 | 96 | ### Phaethontiformes 97 | --Phaethontidae 98 | 99 | ### Podicipediformes 100 | --Podicipedidae 101 | 102 | ### Suliformes 103 | --Sulidae 104 | --Anhingidae 105 | 106 | ---------- 107 | 108 | A secondary set of Orders to enter after the above are finished. When they have been completed, move them into the section above. 109 | 110 | ### Charadriiformes 111 | --Scolopacidae 112 | --Alcidae 113 | --Charadriidae 114 | --Haematopodidae 115 | --Jacanidae 116 | --Laridae (Indigo) 117 | --Recurvirostridae 118 | --Stercorariidae 119 | 120 | ### Procellariiformes (Colleen) 121 | --Diomedeidae 122 | --Procellariidae 123 | --Hydrobatidae 124 | 125 | ### Suliformes 126 | --Fregatidae (Indigo) 127 | -------------------------------------------------------------------------------- /dietdatabase.Rproj: -------------------------------------------------------------------------------- 1 | Version: 1.0 2 | 3 | RestoreWorkspace: Default 4 | SaveWorkspace: Default 5 | AlwaysSaveHistory: Default 6 | 7 | EnableCodeIndexing: Yes 8 | UseSpacesForTab: Yes 9 | NumSpacesForTab: 2 10 | Encoding: UTF-8 11 | 12 | RnwWeave: Sweave 13 | LaTeX: pdfLaTeX 14 | -------------------------------------------------------------------------------- /globi.json: -------------------------------------------------------------------------------- 1 | { 2 | "_comment": "Sample GloBI dataset descriptor. See http://github.com/globalbioticinteractions for more information.", 3 | "format": "hurlbert", 4 | "citation": "Hurlbert, A. H., Olsen, A. M., Sawyer, M. M., and Winner, P. M. 2021. Avian Diet Database. https://doi.org/10.5281/zenodo.5151056", 5 | "resources": { 6 | "AvianDietDatabase.txt": "AvianDietDatabase.txt" 7 | } 8 | } 9 | -------------------------------------------------------------------------------- /in_progress/AvianDietDatabase_Schneider_and_Pence_1992.txt: -------------------------------------------------------------------------------- 1 | Common_Name Scientific_Name Subspecies Family Taxonomy Longitude_dd Latitude_dd Altitude_min_m Altitude_mean_m Altitude_max_m Location_Region Location_Specific Habitat_type Observation_Month_Begin Observation_Year_Begin Observation_Month_End Observation_Year_End Observation_Season Analysis_Number Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Inclusive_Prey_Taxon Prey_Name_ITIS_ID Prey_Name_Status Prey_Stage Prey_Part Prey_Common_Name Fraction_Diet Diet_Type Item_Sample_Size Bird_Sample_Size Sites Study_Type Notes Entered_By Source 2 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Alaska Barrow Tundra 5 1953 8 1953 Summer Lemmus sibiricus yes brown lemmings 1 Unspecified Behavioral observation JF Pitelka 1955a 3 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Massachusetts Nantucket Island 1985 1985 Animalia Chordata Mammalia Rodentia Myomorpha Cricetidae Microtus Microtus pennsylvanicus yes meadow voles 0.933 Items 1214 1 Pellet contents GG Holt and Melvin 1986 4 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Massachusetts Nantucket Island 1986 1987 Spring; Summer Animalia Chordata Mammalia Rodentia Myomorpha Cricetidae Microtus Microtus pennsylvanicus yes meadow voles 0.89 Items 1992 1 Pellet contents GG Tate 1991 5 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Massachusetts Nantucket Island 1986 1987 Spring; Summer Animalia Chordata Mammalia Soricomorpha Soricidae Blarina Blarina brevicauda yes short-tailed shrews 0.03 Items 1992 1 Pellet contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 6 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Massachusetts Nantucket Island 1986 1987 Spring; Summer Animalia Chordata Mammalia Rodentia Myomorpha Cricetidae Peromyscus Peromyscus leucopus yes white-footed mice 0.03 Items 1992 1 Pellet contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 7 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Massachusetts Nantucket Island 1986 1987 Spring; Summer Animalia Chordata Aves yes birds 0.025 Items 1992 1 Pellet contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 8 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Massachusetts Nantucket Island 1986 1987 Spring; Summer Animalia Arthropoda Insecta yes insects 0.01 Items 1992 1 Pellet contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 9 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Massachusetts Nantucket Island 1986 1987 Spring; Summer Animalia Chordata Mammalia no misc. mammals 0.015 Items 1992 1 Pellet contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 10 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 United States New York 1967 1970 Animalia Chordata Mammalia Rodentia Myomorpha Cricetidae Microtus Microtus pennsylvanicus yes meadow voles 0.91 Items 1 Pellet contents GG Clark 1975 11 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Finland 1983 1983 Animalia Chordata Mammalia Rodentia Myomorpha Cricetidae Microtus yes voles 0.789 Items 1 Pellet contents GG Mikkola 1983 12 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Finland 1983 1983 Animalia Chordata Mammalia Clethrionomys no voles and shrews 0.211 Items 1 Pellet contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 13 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Norway 1983 1983 Animalia Chordata Mammalia Rodentia Myomorpha Cricetidae Microtus yes voles 0.651 Items 1 Pellet contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 14 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Norway 1983 1983 Animalia Chordata Mammalia Clethrionomys no voles and shrews 0.349 Items 1 Pellet contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 15 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Germany 1983 1983 Animalia Chordata Mammalia Rodentia Myomorpha Cricetidae Microtus yes voles 0.946 Items 1 Pellet contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 16 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Hungary 1983 1983 Animalia Chordata Mammalia Rodentia Myomorpha Cricetidae Microtus yes voles 0.211 Items 1 Pellet contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 17 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Hungary 1983 1983 Animalia Chordata Mammalia Soricomorpha Soricidae yes shrew 0.699 Items 1 Pellet contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 18 | Short-eared Owl Asio flammeus flammeus Strigidae eBird Clements Checklist v2018 Finland 1977 1977 Animalia Chordata Mammalia Rodentia Myomorpha Cricetidae Microtus yes voles 0.98 Items 1 Pellet contents Includes both M. agrestis and M. arvalis GG Korpimaki 1984 19 | 20 | 21 | Sedge Wren Cistothorus stellaris Troglodytidae eBird Clements Checklist v2018 United States Florida 1932 1932 Animalia Arthropoda no spiders and insects 1 Items 34 1 Stomach contents "Includes spiders, ants, bugs, weevils, ladybird beetles, moths, caterpillars, locusts, crickets, and grasshoppers " GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 22 | Sedge Wren Cistothorus stellaris Troglodytidae eBird Clements Checklist v2018 United States Pennsylvania 1890 1890 Animalia Arthropoda no spiders and beetles 1 Items 7 1 Stomach contents GG "Warren, B. H. 1890. Report of the Birds of Pennsylvania, 2nd ed. Authority of the Commonwealth. " 23 | American bittern Botaurus lentiginosus Ardeidae eBird Clements Checklist v2018 North America Animalia Arthropoda Insecta yes insects 0.23 Items 133 1 Stomach contents "Based on data from Cottam and Uhler 1945, not directly available" GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 24 | American bittern Botaurus lentiginosus Ardeidae eBird Clements Checklist v2018 North America Animalia Chordata Amphibia yes frogs and salamanders 0.21 Items 133 1 Stomach contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 25 | American bittern Botaurus lentiginosus Ardeidae eBird Clements Checklist v2018 North America Animalia Chordata Teleostei yes fish 0.21 Items 133 1 Stomach contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 26 | American bittern Botaurus lentiginosus Ardeidae eBird Clements Checklist v2018 North America Animalia Chordata Crustacea Decapoda yes crayfish 0.19 Items 133 1 Stomach contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 27 | American bittern Botaurus lentiginosus Ardeidae eBird Clements Checklist v2018 North America Animalia Chordata Mammalia Soricomorpha Soricidae yes shrew 0.1 Items 133 1 Stomach contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 28 | American bittern Botaurus lentiginosus Ardeidae eBird Clements Checklist v2018 North America Animalia Chordata Reptilia Squamata Serpentes yes snake 0.05 Items 133 1 Stomach contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 29 | Least Bittern Ixobrychus exilis Ardeidae eBird Clements Checklist v2018 United States Florida Animalia Chordata Teleostei yes fish 0.4 Items 93 1 Stomach contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 30 | Least Bittern Ixobrychus exilis Ardeidae eBird Clements Checklist v2018 United States Florida Animalia Arthropoda Insecta Odonata yes dragonflies 0.21 Items 93 1 Stomach contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 31 | Least Bittern Ixobrychus exilis Ardeidae eBird Clements Checklist v2018 United States Florida Animalia Arthropoda Insecta no aquatic insects 0.12 Items 93 1 Stomach contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 32 | Least Bittern Ixobrychus exilis Ardeidae eBird Clements Checklist v2018 United States Florida Animalia Chordata Crustacea yes crustaceans 0.1 Items 93 1 Stomach contents GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 33 | Black Tern Childonias niger Laridae eBird Clements Checklist v2018 United States New York Animalia Chordata Teleostei Cypriniformes yes minnows 0.41 Items 376 1 behvioral observation "Based on data from Goodwin 1960, not directly available" GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 34 | Black Tern Childonias niger Laridae eBird Clements Checklist v2018 United States New York Animalia Arthropoda Insecta yes insects 0.59 Items 376 1 behvioral observation GG "Schneider, K. J. and Pence, D. M. 1992. Migratory nongame birds of management concern in the Northeast." 35 | 36 | 37 | 38 | 39 | Birds with Usable Data Other papers? 40 | Common Loon n y 41 | Pied-Billed Grebe y? 42 | American Bittern y 43 | Least Bittern y 44 | Northern Harrier n y 45 | Black Rail n y 46 | Gull-Billed Tern n y 47 | Black Tern y 48 | Sedge Wren n m 49 | Seaside Sparrow y 50 | 51 | 52 | Upland Sandpiper n y 53 | Barn Owl y 54 | Loggerhead Shrike n y 55 | Bachman's Sparrow y 56 | Henslow's Sparrow y 57 | 58 | Red-Shouldered Hawk n y 59 | Olive-Sided Flycatcher "yes (beal, check if info has been entered)" 60 | Golden-Winged Warbler n 61 | Cerulean Warbler y -------------------------------------------------------------------------------- /in_progress/fakeclean.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/in_progress/fakeclean.txt -------------------------------------------------------------------------------- /in_progress/reshape_Stillman_et_al_raw_data.r: -------------------------------------------------------------------------------- 1 | library(dplyr) 2 | library(stringr) 3 | 4 | 5 | otu_dat = read.csv('OTU_dataset.csv', header = F) %>% 6 | rename(OTU_ID = V1) 7 | 8 | 9 | otu_tax = read.csv('OTU_taxonomy.csv', header = T) %>% 10 | mutate_all(function(x) str_replace(x, "N/A","NA")) %>% 11 | mutate(Prey_Kingdom = str_replace(kingdom, "k:", ""), 12 | Prey_Phylum = str_replace(phylum, "p:", ""), 13 | Prey_Class = str_replace(class, "c:", ""), 14 | Prey_Order = str_replace(order, "o:", ""), 15 | Prey_Suborder = NA, 16 | Prey_Family = str_replace(family, "f:", ""), 17 | Prey_Genus = str_replace(genus, "g:", ""), 18 | Prey_Scientific_Name = str_replace(species, "s:", "")) %>% 19 | select(OTU_ID, Prey_Kingdom, Prey_Phylum, Prey_Class, Prey_Order, Prey_Suborder, 20 | Prey_Family, Prey_Genus, Prey_Scientific_Name) 21 | 22 | otu_join = left_join(otu_dat, otu_tax, by = 'OTU_ID') %>% 23 | select(Prey_Kingdom:Prey_Scientific_Name, V2:V75) 24 | 25 | write.csv(otu_join, 'Stillman_et_al_2022_raw_woodpecker_data.csv', row.names = F) 26 | -------------------------------------------------------------------------------- /instructions/cloning_repo_with_RStudio.md: -------------------------------------------------------------------------------- 1 | ## How to work with the Avian Diet Database on your own machine 2 | 3 | 1. Make sure you have both [R](https://mirrors.nics.utk.edu/cran/) and [RStudio](https://rstudio.com/products/rstudio/download/#download) installed on your machine. 4 | 2. Open RStudio. 5 | 3. File > New Project > Version Control > Git. In the “repository URL” paste the URL of the Avian Diet Database GitHub repository. https://github.com/hurlbertlab/dietdatabase/. 6 | * Do you NOT see an option to get the Project from Version Control? Make sure RStudio can find Git. 7 | 4. Decide where to store the local directory for the Project. Don’t scatter everything around your computer - have a central location, or some meaningful structure. 8 | 5. I suggest you check “Open in new session”, as that’s what you’ll usually do in real life. 9 | 6. Click “Create Project” to create a new sub-directory, which will be all of these things: 10 | * a directory on your computer 11 | * a Git repository, linked to a remote GitHub repository 12 | * an RStudio Project 13 | 7. You should now be able to operate just as you have from the lab computers. -------------------------------------------------------------------------------- /instructions/git_dietdatabase_help.md: -------------------------------------------------------------------------------- 1 | ## Data Entry and Versioning using Git 2 | 3 | The database is under version control using Git so that we can easily go back to previous states, it's automatically 4 | backed up, and many people can access it and add records simultaneously from different computers. 5 | 6 | This means that you will need to learn some basic Git commands for working with it. 7 | 8 | ### Getting a ssh key 9 | It may be useful to get a ssh key which helps Git know that you and the machine you are working on are both valid. This 10 | is something you only need to do once (per machine that you work on), probably when you first get set up. Follow the 11 | instructions [here](ssh_instructions.md). 12 | 13 | 14 | ### pull, add, commit, push 15 | From your local machine, open Git (e.g. using Git Bash from a Windows machine), and 'pull' down the most up-to-date 16 | version of the database after making sure you're in the right directory housing the repository. 17 | 18 | ``` 19 | $ cd /c/git/dietdatabase 20 | $ git pull origin master 21 | ``` 22 | 23 | Once you've encountered a study with quantitative diet data to enter, you want to open the file 'AvianDietDatabase_template.txt' in Excel and re-save it, replacing the word 'template' with the study author and year, e.g. 'AvianDietDatabase_Beaver_and_Baldwin_1975.txt'. Now you need to tell Git that this is a file you want to keep track of. We do 24 | this using `git add` like this: 25 | 26 | ``` 27 | $ git add AvianDietDatabase_Beaver_and_Baldwin_1975.txt 28 | ``` 29 | 30 | You can now begin entering data as described on the [main Instructions page](https://github.com/hurlbertlab/dietdatabase/blob/master/instructions/instructions.md). When you are finished with data entry for the day, be sure to Save As a tab-delimited .txt file (with the same name, in the same folder). 31 | 32 | Now you need to stage your committed changes, add a descriptive message of what you've added, and 'push' the new version 33 | to the master repository. 34 | 35 | ``` 36 | $ git commit -am "added 3 diet records for red-eyed vireo and 2 for white-eyed vireo" 37 | $ git push origin master 38 | ``` 39 | 40 | Enter your github userid and password if prompted. Now your up-to-date files are available for incorporation into GloBI and 41 | for others to add to! 42 | 43 | ## Managing Git through RStudio 44 | Assuming you have Git installed, you can clone the *Diet Database* repository by opening RStudio and clicking on _File_ > _New Project_ > _Version Control_ > _Git_. Under _Repository URL_ enter `https://github.com/hurlbertlab/dietdatabase`, leave _Project directory name_ blank, and then specify where on your local machine you would like to install the project folder. Then click _Create Project_. 45 | 46 | From now on, when you want to work on this project, navigate to the local folder you just created above, and open the `dietdatabase.Rproj` file which will open up in RStudio. 47 | 48 | * Click on the _Git_ tab in the upper right. 49 | * When working with this repo, ALWAYS begin your session by clicking _Pull_ (the equivalent of `git pull origin master`) to make sure you have the most up-to-date version of the repo on your local machine. 50 | * Anytime you modify a file in the repository, it will be listed in this _Git_ panel. 51 | * Checking the box next to the filename is the equivalent of `git add`. 52 | * Once you've checked the boxes of the modified files you want to add, click the _Commit_ and provide a brief description of your edits. Click the _Commit_ button. 53 | * Finally, click _Push_ (the equivalent of `git push origin master`). If there are no error messages, your changes have been pushed to the master version on Github! 54 | 55 | ## Potential Problems 56 | Occasionally, when you try to push your latest changes, you will get an error like this: 57 | 58 | ``` 59 | error: failed to push some refs to 'https://github.com/hurlbertlab/dietdatabase.git' 60 | hint: Updates were rejected because the remote contains work that you do 61 | hint: not have locally. This is usually caused by another repository pushing 62 | hint: to the same ref. You may want to first integrate the remote changes 63 | hint: (e.g., 'git pull ...') before pushing again. 64 | hint: See the 'Note about fast-forwards' in 'git push --help' for details. 65 | ``` 66 | 67 | This comes up when two people both download the latest version of the database, and each 68 | person makes some changes (i.e., adds some data) independent of the other. The second 69 | person to try to push their changes doesn't have the changes made by the first, and so 70 | Git points out the potential problem. 71 | 72 | As the hint message suggests, try re-pulling the repository with a `$ git pull origin master`. 73 | (Note that you need to have closed the database file on your machine before pulling, 74 | or you will get a message that your file is "unlinked". In that case, just close the file 75 | and try again.) Usually, this will take care of the problem and you will see a message like this: 76 | 77 | ``` 78 | Auto-merging AvianDietDatabase.txt 79 | Merge made by the 'recursive' strategy. 80 | ``` 81 | 82 | In that case, Git figured out that you both were making changes to different parts of the 83 | database, and incorporated both sets of changes to the most up to date version. 84 | 85 | However, if you were both editing the same part of the database (and this 86 | includes a scenario where both people are simply adding different data to the 87 | bottom of the file), you might see this message after you try to pull. 88 | 89 | ``` 90 | Auto-merging AvianDietDatabase.txt 91 | CONFLICT (content): Merge conflict in AvianDietDatabase.txt 92 | Automatic merge failed; fix conflicts and then commit the result. 93 | ``` 94 | 95 | In this case, Git couldn't figure out what to do, so you will have to resolve the problem 96 | manually. Open the file that has the merge conflict like you normally would. Git 97 | has flagged the conflict within the region, so search (Ctrl-F) for the following 98 | text: "<<<<<<< HEAD" (without the quotes). Now let's say you just added a row of data 99 | for American Robin as the last line in the database, and someone else added a row of 100 | data for Red-winged Blackbird. Then you will see something like this: 101 | 102 | ``` 103 | <<<<<<< HEAD 104 | American Robin (and whatever other data is in this line) 105 | ======== 106 | Red-winged Blackbird (and whatever other data is in THIS line) 107 | >>>>>>>dca3kdjs33jdj3 108 | ``` 109 | 110 | Everything above the ======= line is one version, and everything below is the other 111 | version. In this case, you want both of the edits to be saved in the final version, 112 | so simply delete the entire lines starting with <<<<<<<, ========, and >>>>>>>. 113 | 114 | ``` 115 | American Robin (and whatever other data) 116 | Red-winged Blackbird (and whatever other data) 117 | ``` 118 | 119 | Then save the file, commit the change with a short message, and push it as you 120 | normally would. There should be no error messages! 121 | 122 | In the event that both of you actually edited the same line in the database, and 123 | one of the versions is out of date or incorrect, then you would simply delete 124 | the edits you did not want to keep in addition to the <<<<<<, =======, and >>>>>>> lines. 125 | 126 | ``` 127 | <<<<<<< HEAD 128 | American Robin some wrong data on this line 129 | ======= 130 | American Robin some correct data, or edits that you want to keep 131 | >>>>>>>dca83kd9sfas933ks33 132 | ``` 133 | 134 | gets edited down to just 135 | 136 | ``` 137 | American Robin some correct data, or edits that you want to keep 138 | ``` 139 | 140 | Save. Commit. Push. 141 | 142 | 143 | -------------------------------------------------------------------------------- /instructions/ssh_instructions.md: -------------------------------------------------------------------------------- 1 | # Generating a new ssh key on a Windows machine 2 | 3 | 1. In a git bash window, type: 4 | 5 | ``` 6 | $ ssh-keygen -t rsa -C "your_email@example.com" 7 | ``` 8 | 9 | 2. You'll be asked to enter your ssh directory and a passphrase. Just leave these blank and hit enter for each selection. The location of your ssh key and the key fingerprint are printed. The location should be in your home directory. 10 | 3. Type `ls .ssh` to be shown the ssh files. _Note: You can type_ `pwd` _to print the working directory location. 11 | 4. Navigate to the location of the ssh key in **Windows Explorer** (should be in the home folder of your user name). 12 | 5. Open the ssh file, *id_rsa.pub* (There will be two id_rsa files, if you cannot see the extension, it is the one listed as a Microsoft Publisher Document), in **notepad**. 13 | 6. Copy the *entire* contents of the file (Ctrl+A, Ctrl+C) 14 | 7. In your **web browser**, navigate to your GitHub account online. 15 | * Click the settings button (upper right-hand corner of your screen, looks like a bicycle sprocket). 16 | * Click the SSH keys menu option (under the personal settings). 17 | * Click the "Add SSH key". 18 | * Provide a title for the key (e.g., "Allens laptop"). 19 | * Paste the **entire** contents of your clipboard in the "key" field and click the "Add key" button. 20 | 8. Navigate back to your **git bash** window. 21 | 9. Test out whether you've successfully connected the key by typing (don't worry if there's a warning): 22 | 23 | ```ssh -T git@github.com``` 24 | 25 | -------------------------------------------------------------------------------- /instructions/training_instructions.md: -------------------------------------------------------------------------------- 1 | INSTRUCTIONS FOR FINDING AND ENTERING AVIAN DIET DATA 2 | ===================================================== 3 | 4 | 1. Use the "NA_avian_diet_refs.txt" file to identify potential studies with diet 5 | information for the species you are working on. 6 | 7 | 2. Find the study by pasting the title into Google Scholar or Web of Science. 8 | 9 | 3. Search for tabular, quantitative diet data. In some cases, it's possible 10 | that quantitative diet data is described in the text and a table is not provided 11 | but this should be rare. 12 | 13 | 4. Make sure you understand which bird species is being studied. Many species 14 | names (both common names and scientific names) have changed over time, so older 15 | studies in particular may be using one that is out of date. Check whether the 16 | common and scientific names of your bird are listed in the [most recent eBird 17 | taxonomy checklist](birdtaxonomy/eBird_Taxonomy_v2021.csv) in the `birdtaxonomy` 18 | folder (try using Ctrl-F or Cmd-F to search for a name). If both names are present 19 | in the eBird checklist then you can enter them as is into the database file. 20 | 21 | If the name is not in the eBird taxonomy checklist, try pasting the scientific 22 | name into the search bar at http://avibase.bsc-eoc.org. If a "Search results" 23 | page comes up, click on the first link for which "Status" is blank and for 24 | which there is no additional subspecies name listed. This should bring you to an 25 | information page for this taxon. Click on the "taxon grid" link across the top, 26 | and then on "American Ornithologists' Union - versions 1 to 7". In general, we 27 | will be using the scientific and common names according to the most recent 28 | edition (currently the 7th) of the American Ornithologists' Union taxonomy. 29 | 30 | Try this for the following species and see what happens: 31 | 32 | *Dendroica townsendi*. Note that this search goes straight to the taxon page. 33 | The taxon grid is quite straightforward, and shows that there is a single 34 | species concept which has changed names over time from *Dendroica townsendi* to 35 | *Setophaga townsendi*. Re-check the eBird Taxonomy checklist for this name, and 36 | you'll find it's present--that's the name that should be used in the database 37 | even if the study the data are coming from called it the former. 38 | 39 | *Empidonax difficilis*. This one is more complicated, with many more species 40 | concepts listed on [Avibase](http://avibase.bsc-eoc.org/species.jsp?avibaseid=44A2028364A252A6&sec=taxontable&version=aou). 41 | Many of these are different subspecies which we can ignore 42 | (e.g., the bottom two rows, and 3 of the middle rows). Otherwise, what this 43 | taxon grid shows is that up through the 6th edition (1983), there was a single 44 | species concept referred to as *Empidonax difficilis* with the common name 45 | "Western Flycatcher". However, by 1998 and the publication of the 7th edition, 46 | taxonomists had split this species into two separate species. One of them 47 | retained *Empidonax difficilis* as its scientific name, but the common name 48 | changed to "Pacific-slope Flycatcher". The other took on the scientific name 49 | *Empidonax occidentalis* and the common name "Cordilleran Flycatcher". 50 | 51 | If you came across a study reporting diet info for *Empidonax difficilis*, 52 | which species is it referring to? If it's an older study, it could be referring 53 | to either one. One way to figure this out is by checking the geographic ranges 54 | of the two split species. Click on the "eBird" link across the top of the Avibase page 55 | to see where "Pacific-slope Flycatcher" (i.e. the concept that *Empidonax 56 | difficilis* currently refers to) occurs. Take a look, and then in the "Related 57 | taxa" dropdown menu, select "*Empidonax occidentalis*" to see where "Cordilleran 58 | Flycatcher" occurs. There is some overlap, but if the study was done in 59 | California, they were probably referring to Pacific-slope Flycatcher. If it was 60 | done in Colorado, it was probably referring to Cordilleran Flycatcher. Usually 61 | (but not always!) this information will help you narrow down which species 62 | the study actually refers to. 63 | 64 | 5. Read the study and enter as much information as you can find about where 65 | (latitude, longitude, altitude, regions, place names) and when (years, season) 66 | it was conducted. Refer to the https://github.com/hurlbertlab/dietdatabase 67 | README file for more details. 68 | 69 | 6. Read the table caption and/or Methods section carefully to understand how 70 | the study quantified diet. This will determine what you specify as the "Diet_Type". The options are: 71 | 72 | 73 | 74 | 75 | 78 | 79 | 80 | 81 | 85 | 86 | 87 | 88 | 90 | 91 | 92 | 93 | 95 | 96 | 97 |
Wt_or_VolFraction of the diet as measured by weight or volume. E.g., all beetles 76 | in the stomach contents were weighed, and this value was divided by the mass 77 | of all stomach contents. Convert all %s to fractions between 0 and 1.
ItemsFraction of the diet as measured by a count of the number of prey items. 82 | E.g., the number of beetles in the stomach contents were counted, and this 83 | value was divided by the total number of unique prey items in the stomach 84 | contents. Convert all %s to fractions between 0 and 1.
OccurrenceFraction of the birds examined that contained at least one individual of 89 | this prey type. Convert all %s to fractions between 0 and 1.
UnspecifiedFraction of the diet of the prey item based on a methodology unspecified by 94 | the authors. Convert all %s to fractions between 0 and 1.
98 | 99 | In some cases, the data will not be provided as a fraction or %, and you will 100 | need to calculate this yourself. If so, make a note about what you did in the 101 | Notes field. If you are calculating fractions yourself, **round entries to the 102 | nearest 0.001**. If a value is reported as "<0.01", then **DO NOT ENTER the "<" sign** 103 | but instead, enter one half of the upper limit reported (in this example, 0.005). 104 | 105 | 7. When entering information on the prey, try to fill in all of the higher taxonomic level 106 | information above the taxonomic level reported. For example, if the prey category reported is the Order "Coleoptera" 107 | (i.e. beetles), then you would also fill in "Animalia", "Arthropoda", and "Insecta" for the 108 | Prey_Kingdom, Prey_Phylum, and Prey_Class fields. You can verify this information by pasting 109 | the prey name into the [Global Names Resolver](http://resolver.globalnames.org/) and 110 | clicking on "Resolve Names". This will generate a report of the hierarchical classification 111 | of your name as reported by various entities. For consistency, we will typically rely on the 112 | reported classification of [ITIS](https://itis.gov) (Integrated Taxonomic Information System), which in 113 | this example provides a report that looks like this (try finding "ITIS" on the browser page 114 | using Ctrl-F as it may be a ways down in the report): 115 | 116 | Coleoptera Linnaeus, 1758 [ exact canonical match, Score: 0.75 ] 117 | ITIS 118 | Animalia (Kingdom) >> Bilateria (Subkingdom) >> Protostomia (Infrakingdom) >> Ecdysozoa (Superphylum) >> Arthropoda (Phylum) >> Hexapoda (Subphylum) >> Insecta (Class) >> Pterygota (Subclass) >> Neoptera (Infraclass) >> Holometabola (Superorder) >> Coleoptera (Order) 119 | 120 | This provides many intermediate levels of the taxonomic hierarchy, and for our purposes we are just focusing on 121 | Kingdom, Phylum, Class, Order, Suborder (if listed), Family, Genus, or full Scientific Name. 122 | 123 | 8. In some cases, the prey name as given in the paper will not match any currently accepted 124 | ITIS name. Sometimes you may find an indication that the name has been changed, and you 125 | can see if that changed name is recognized by ITIS. If so, go ahead and enter the valid ITIS 126 | name in the database. If you cannot find a valid name, simply enter the name as reported. 127 | 128 | --- 129 | 130 | Try entering data from the two studies listed below. Open the AvianDietDatabase_template.txt 131 | template in Excel and then Save into the same folder using a new file name where you add 132 | your initials to the end (e.g. 'training_dietdatabase_AHH.txt'). 133 | 134 | # Training Dataset 1 135 | 136 | Beaver and Baldwin 1975. Ecological overlap and the problem of competition and sympatry 137 | in the Western and Hammond's Flycatchers. Condor 77: 1-13. 138 | 139 | Look out for any taxonomic issues! 140 | 141 | # Training Dataset 2 142 | 143 | Allaire, P. N. and C. D. Fisher. 1975. Feeding ecology of three resident sympatric 144 | sparrows in eastern Texas. Auk 92:260-269. 145 | 146 | Note here that diet information is provided for different seasons, which should be 147 | entered separately. 148 | 149 | In addition, percentages given are for seeds and arthropods separately such that 150 | each group adds to 100%. If you want to characterize the stomach contents so that 151 | seeds plus arthropods adds up to 100%, how would you do it? Consider this example, 152 | where you know that there were 110 total seeds found in the stomach contents, and 30 153 | total insect individuals. 154 | 155 | | Diet item | Percent | 156 | |----------|---------| 157 | | Seed 1 | 80 | 158 | | Seed 2 | 20 | 159 | | Insect 1 | 60 | 160 | | Insect 2 | 40 | 161 | 162 | Try to work it out for yourself. 163 | 164 | You should have gotten 62.9% of Seed 1 (.8 * 110) / (110 + 30), 165 | 15.7% of Seed 2 (.2 * 110) / (110 + 30), 166 | 12.9% of Insect 1 (.6 * 30) / (110 + 30), and 167 | 8.6% of Insect 2 (.4 * 30) / (110 + 30). 168 | -------------------------------------------------------------------------------- /scripts/bird_species_list.r: -------------------------------------------------------------------------------- 1 | # Checklist of birds of the United States and Canada obtained from Avibase here: 2 | # https://avibase.bsc-eoc.org/checklist.jsp?lang=EN&p2=1&list=clements&synlang=®ion=NA1&version=text&lifelist=&highlight=0 3 | 4 | # Checklist requires copying from html and cleaning out blank and Order:Family header rows, 5 | # then saving as Clements_Taxonomy_v20XX_NorthAmerica.csv 6 | 7 | 8 | #Description: All 50 US States, Canada, St-Pierre & Miquelon 9 | #Number of species: 1237 10 | #Number of endemics: 26 11 | #Number of breeding endemics: 1 12 | #Number of globally threatened species: 115 13 | #Number of extinct species: 30 14 | #Number of introduced species: 133 15 | 16 | # Read this file in, then weed out all species that are Extinct, Rare/Accidental, or Introduced. 17 | 18 | library(dplyr) 19 | library(stringr) 20 | 21 | birdtax = read.csv('birdtaxonomy/Clements_Taxonomy_v2018_NorthAmerica.csv', header=T, 22 | quote = '', fill = T, stringsAsFactors = FALSE) 23 | 24 | exclude = c("Introduced", "Extinct", "Extirpated", "Rare/Accidental") 25 | 26 | btax = filter(birdtax, !grepl(paste(exclude, collapse = "|"), Status)) 27 | 28 | # Add a few species that are introduced but long enough ago to be considered part of the avifauna 29 | add = filter(birdtax, CommonName %in% c("Rock Pigeon", "House Sparrow", "Alder/Willow Flycatcher (Traill's Flycatcher)", 30 | "Mute Swan", "Ring-necked Pheasant", "European Starling")) 31 | 32 | tax = rbind(btax, add) 33 | 34 | # Merge in Family and Order names from overall eBird 2018 checklist 35 | # available here: http://www.birds.cornell.edu/clementschecklist/download/ 36 | ebird = read.table('birdtaxonomy/eBird_Taxonomy_v2018.csv', header = T, 37 | sep = ',', quote = '\"', stringsAsFactors = F) 38 | 39 | spplist = left_join(tax, ebird, by = c("CommonName" = "PRIMARY_COM_NAME")) %>% 40 | mutate(Family = word(FAMILY, 1), list = 1) %>% 41 | select(CommonName, SciName, Family, ORDER1, list) 42 | 43 | # Now compare to species in the Diet Database 44 | diet = read.table('AvianDietDatabase.txt', header=T, sep = '\t', quote = '', fill = T, stringsAsFactors = FALSE) 45 | 46 | dietsp = unique(diet[, c('Common_Name', 'Scientific_Name', 'Family')]) 47 | names(dietsp) = c('CommonName', 'SciName', 'Family') 48 | dietsp$dietdb = 1 49 | 50 | spplist2 = full_join(spplist, dietsp, by = c('CommonName', 'SciName', 'Family')) 51 | 52 | spplist2$list[is.na(spplist2$list)] = 0 53 | spplist2$dietdb[is.na(spplist2$dietdb)] = 0 54 | 55 | 56 | famtotals = spplist2 %>% group_by(ORDER1, Family) %>% 57 | summarize(TotalSp = sum(list), WithData = sum(dietdb)) %>% 58 | mutate(WithoutData = TotalSp - WithData) %>% 59 | data.frame() 60 | 61 | DBnamesNotInChecklist = spplist2 %>% 62 | filter(dietdb == 1, list == 0, 63 | !CommonName %in% c('Turkey/Black Vulture', "Alder/Willow Flycatcher (Traill's Flycatcher)", 64 | 'Eskimo Curlew', 'Passenger Pigeon', 'Carolina Parakeet', 'Chukar', 65 | 'Gray Partridge', 'Yellow-billed/Black-billed Cuckoo')) %>% 66 | left_join(ebird[, c('PRIMARY_COM_NAME', 'SCI_NAME')], by = c('CommonName' = 'PRIMARY_COM_NAME')) %>% 67 | select(CommonName, SciName, SCI_NAME, Family, dietdb, list) 68 | 69 | 70 | -------------------------------------------------------------------------------- /scripts/database_summary_functions.R: -------------------------------------------------------------------------------- 1 | # Data summaries of the Avian Diet Database 2 | 3 | library(dplyr) 4 | library(stringr) 5 | library(tidyr) 6 | 7 | 8 | ################# 9 | 10 | dbSummary = function(diet = NULL) { 11 | # Silence dplyr summarize warning 12 | options(dplyr.summarise.inform = FALSE) 13 | 14 | if (is.null(diet)) { 15 | diet = read.table('aviandietdatabase.txt', header=T, sep = '\t', quote = "\"", 16 | fill=T, stringsAsFactors = F) 17 | } 18 | speciesList = read.csv('birdtaxonomy/NA_specieslist.csv', header = T, quote = '\"') 19 | 20 | dbSpecies = unique(diet[, c('Common_Name', 'Family')]) 21 | dbSpecies$db = 1 22 | numSpecies = nrow(dbSpecies) 23 | numStudies = length(unique(diet$Source)) 24 | numRecords = nrow(diet) 25 | 26 | analysesPerSpecies = diet %>% 27 | group_by(Common_Name) %>% 28 | summarize(analyses = n_distinct(Longitude_dd, 29 | Latitude_dd, Altitude_min_m, Altitude_mean_m, Altitude_max_m, 30 | Location_Region, Location_Specific, Habitat_type, 31 | Observation_Month_Begin, Observation_Year_Begin, 32 | Observation_Month_End, Observation_Year_End, Observation_Season, 33 | Analysis_Number, Source)) 34 | 35 | recordsPerSpecies = count(diet, Common_Name) %>% 36 | as_tibble() %>% 37 | left_join(analysesPerSpecies, by = 'Common_Name') %>% 38 | rename(records = n) 39 | 40 | 41 | familyCoverage = left_join(speciesList, dbSpecies, by = c('common_name' = 'Common_Name', 'family' = 'Family')) %>% 42 | group_by(order, family) %>% 43 | summarize(SpeciesWithData = sum(db, na.rm = TRUE), 44 | SpeciesWithoutData = sum(is.na(db)), 45 | PercentComplete = round(100*SpeciesWithData/(SpeciesWithData + SpeciesWithoutData))) %>% 46 | rename(Order = order, Family = family) 47 | 48 | return(list(numRecords=numRecords, 49 | numSpecies=numSpecies, 50 | numStudies=numStudies, 51 | recordsPerSpecies=recordsPerSpecies, 52 | speciesPerFamily = familyCoverage)) 53 | } 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | # Re classify diet database to a different taxonomic level (of prey). 62 | # Only returns results for Diet_Type 'Items', 'Wt_or_Vol', or 'Unspecified' since 63 | # 'Occurrence' cannot be summed hierarchically. 64 | 65 | reclassifyPrey = function(diet = NULL, by = 'Order') { 66 | if (is.null(diet)) { 67 | diet = read.table('aviandietdatabase.txt', header=T, sep = '\t', quote = '\"', 68 | fill=T, stringsAsFactors = F) 69 | } 70 | 71 | if (by == 'Species') { by = 'Scientific_Name' } 72 | 73 | if (!by %in% c('Kingdom', 'Phylum', 'Class', 'Order', 'Suborder', 74 | 'Family', 'Genus', 'Scientific_Name')) { 75 | warning("Please specify one of the following taxonomic levels to aggregate prey data:\n Kingdom, Phylum, Class, Order, Suborder, Family, Genus, or Scientific_Name") 76 | return(NULL) 77 | } 78 | 79 | dietsp = filter(diet, Diet_Type != 'Occurrence') 80 | 81 | if (nrow(dietsp) == 0) { 82 | warning("No available records with a DietType of 'Items', 'Wt_or_Vol', or 'Unspecified' to reclassify.") 83 | return(NULL) 84 | } 85 | 86 | taxonLevel = paste('Prey_', by, sep = '') 87 | 88 | # If prey not identified down to taxon level specified, replace "" with 89 | # "Unidentified XXX" where XXX is the lowest level specified (e.g. Unidentified Animalia) 90 | dietprey = dietsp[, c('Prey_Kingdom', 'Prey_Phylum', 'Prey_Class', 91 | 'Prey_Order', 'Prey_Suborder', 'Prey_Family', 92 | 'Prey_Genus', 'Prey_Scientific_Name')] 93 | level = which(names(dietprey) == taxonLevel) 94 | dietsp[, taxonLevel] = apply(dietprey, 1, function(x) 95 | if(x[level] == "" | is.na(x[level])) { paste("Unid.", x[max(which(x != "")[which(x != "") < level], na.rm = T)])} 96 | else { x[level] }) 97 | 98 | # Prey_Stage should only matter for distinguishing things at the Order level and 99 | # below (e.g. distinguishing between Lepidoptera larvae and adults). 100 | if (by %in% c('Order', 'Family', 'Genus', 'Scientific_Name')) { 101 | stage = dietsp$Prey_Stage 102 | stage[is.na(stage)] = "" 103 | stage[stage == 'adult'] = "" 104 | dietsp$Taxon = paste(dietsp[, taxonLevel], stage) %>% trimws("both") 105 | } else { 106 | dietsp$Taxon = dietsp[, taxonLevel] 107 | } 108 | 109 | TaxonLevelAbove = names(dietprey)[level - 1] 110 | 111 | # Summarizing by new taxonomic level 112 | reclassified = dietsp %>% 113 | group_by(Common_Name, Scientific_Name, Subspecies, Family, Taxonomy, Longitude_dd, Latitude_dd, 114 | Altitude_min_m, Altitude_mean_m, Altitude_max_m, Location_Region, Location_Specific, 115 | Habitat_type, Observation_Month_Begin, Observation_Year_Begin, Observation_Month_End, 116 | Observation_Year_End, Observation_Season, Prey_Kingdom, get(TaxonLevelAbove, envir = as.environment(dietsp)), 117 | get(taxonLevel, envir = as.environment(dietsp)), Diet_Type, Item_Sample_Size, Bird_Sample_Size, 118 | Sites, Study_Type, Source) %>% 119 | summarize(Frac_Diet = sum(Fraction_Diet, na.rm = T)) %>% 120 | select(Common_Name, Scientific_Name, Subspecies, Family, Taxonomy, Longitude_dd, Latitude_dd, 121 | Altitude_min_m, Altitude_mean_m, Altitude_max_m, Location_Region, Location_Specific, 122 | Habitat_type, Observation_Month_Begin, Observation_Year_Begin, Observation_Month_End, 123 | Observation_Year_End, Observation_Season, Prey_Kingdom, "get(TaxonLevelAbove, envir = as.environment(dietsp))", 124 | "get(taxonLevel, envir = as.environment(dietsp))", Frac_Diet, Diet_Type, Item_Sample_Size, 125 | Bird_Sample_Size, Sites, Study_Type, Source) 126 | names(reclassified)[names(reclassified) == "get(TaxonLevelAbove, envir = as.environment(dietsp))"] = TaxonLevelAbove 127 | names(reclassified)[names(reclassified) == "get(taxonLevel, envir = as.environment(dietsp))"] = taxonLevel 128 | 129 | reclassified = as.data.frame(reclassified) 130 | return(reclassified) 131 | } 132 | 133 | 134 | 135 | 136 | # For dates with no clear Observation_Year_End, replace 137 | # Observation_Year_End with the publication year. 138 | # (rapply is to exclude any years in the article title) 139 | 140 | fill_study_years = function(diet) { 141 | fixed = diet %>% mutate(pubyear = str_match_all(Source, "[0-9][0-9][0-9][0-9]") %>% 142 | rapply(function(x) head(x, 1)) %>% as.numeric()) %>% 143 | mutate(Observation_Year_End = ifelse(is.na(Observation_Year_End), pubyear, Observation_Year_End)) %>% 144 | select(Common_Name:Source) 145 | return(fixed) 146 | } 147 | 148 | -------------------------------------------------------------------------------- /scripts/name_cleaning_test_data.txt: -------------------------------------------------------------------------------- 1 | Common_Name Scientific_Name Subspecies Family Taxonomy Longitude_dd Latitude_dd Altitude_min_m Altitude_mean_m Altitude_max_m Location_Region Location_Specific Habitat_type Observation_Month_Begin Observation_Year_Begin Observation_Month_End Observation_Year_End Observation_Season Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Unidentified Prey_Name_Status Prey_Stage Prey_Part Prey_Common_Name Fraction_Diet Diet_Type Item_Sample_Size Bird_Sample_Size Sites Study_Type Notes Entered_By Source 2 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 2 1947 4 1947 Spring a 1 whatwhat Ericales NA Ericaceae NA NA yes NA seed Seeds 0.048 Wt_or_Vol NA 21 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 3 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 2 1947 4 1947 Spring b 2 whatwhat Ericales NA Ericaceae yes NA seed Seeds 0.048 Occurrence NA 21 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 4 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 5 1947 7 1947 Summer c 3 whatwhat Ericales NA Arctostaphylos uva-ursi yes NA flower Blossoms 0.027 Wt_or_Vol NA 197 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 5 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 5 1947 7 1947 Summer d 4 whatwhat Ericales NA NA NA NA yes NA seed Seeds 0.02 Wt_or_Vol NA 197 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 6 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 5 1947 7 1947 Summer e 5 whatwhat Ericales NA yes NA flower Blossoms 0.025 Occurrence NA 197 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 7 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 5 1947 7 1947 Summer f Tracheophyta NA yes NA seed Seeds 0.066 Occurrence NA 197 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 8 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 8 1947 10 1947 Fall g Tracheophyta NA NA NA NA NA NA yes NA seed Seeds 0.012 Wt_or_Vol NA 207 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 9 | Band-tailed Pigeon Patagioenas fasciata NA Columbidae "AOU 7th ed., 57th supplement" NA NA NA NA NA United States NA multiple 8 1947 10 1947 Fall Plantae Tracheophyta whatwhat uhuh NA noway Arctostaphylos NA yes NA seed Seeds 0.005 Occurrence NA 207 NA stomach contents NA SS "Neff, J. A. 1947. Habits, food, and economic status of the Band-tailed Pigeon. U.S. Fish Wildl. Serv. N. Am. Fauna 58." 10 | Western Scrub-Jay Aphelocoma californica NA Corvidae "AOU 7th ed., 54th supplement" NA NA NA NA NA California NA agriculture 1 NA 12 NA NA Plantae NA no NA fruit manzanita 0.012 Occurrence NA 326 NA stomach contents "article has common name listed as California Jay, but current common name is Western Scrub-jay" RQ "Beal, F. E. L. 1910. Birds of California in relation to fruit industry, Pt. 2. U.S. Dept. Agricult. Biol. Surv. Bull. 34:1-96." 11 | California Quail Callipepla californica NA Odontophoridae "AOU 7th ed., 54th supplement" NA NA NA NA NA California NA agriculture 1 NA 12 NA NA Plantae NA NA NA NA NA NA NA no NA fruit manzanita 0.003 Occurrence NA 619 NA stomach contents "specimens were collected in every month except for May; study does not give year, although it can be assumed to be close to or before 1927" RQ "Beal, F. E. L. 1910. Birds of California in relation to fruit industry, Pt. 2. U.S. Dept. Agricult. Biol. Surv. Bull. 34:1-96." 12 | Acorn Woodpecker Melanerpes formicivorus NA Picidae "AOU 7th ed., 54th supplement" NA NA NA NA NA Oregon Rogue River Valley and Willamette Valley agriculture; wetland; woodland 11 1924 3 1926 All Gooberycop whatwhat Ericales NA NA NA NO verified NA fruit manzanitas 0.063463282 Wt_or_Vol NA 72 NA stomach contents "data originally included grit/mineral matter as percent of stomach volume, but it was taken out and only the total volume animal/plant were set as the measure of total Wt_or_Vol" JB "Neff, J. A. 1928. A study of the economic status of the common woodpeckers in relation to Oregon horticulture. Free Press Print, Marionville, MO." 13 | Spruce Grouse Falcipennis canadensis NA Phasianidae eBird Clements Checklist v2016 NA NA NA NA NA Montana Whitefish Range NA 9 1960 10 1962 NA Gooberycop Streptophyta whatwhat uhuh NA Ericaceae Arctostaphylos Arctostaphylos uva-ursi NA NA NA fruit NA 0.001 Wt_or_Vol 29 56 NA stomach contents NA AMO "Jonkel, C.J. & Greer, K.R. (1963) Fall food habits of spruce grouse in northwest Montana. Journal of Wildlife Management, 27, 593-596." 14 | test test test NA NA NA NA NA NA NA NA NA NA NA NA NA NA Animalia Arthropoda Insecta Coleoptera 15 | test test test NA NA NA NA NA NA NA NA NA NA NA NA NA NA Animalia Arthropoda Insecta 16 | test test test NA NA NA NA NA NA NA NA NA NA NA NA NA NA Animalia Arthropoda Insecta 17 | test test test NA NA NA NA NA NA NA NA NA NA NA NA NA NA Animalia Arthropoda Insecta 18 | -------------------------------------------------------------------------------- /scripts/prey_common_names.r: -------------------------------------------------------------------------------- 1 | # Link 2 | 3 | # Common names from iNat database (through 2018) as hosted on Hurlbert Lab server 4 | # assigning kingdoms based on iconic_taxon_name 5 | icon_kingdom = data.frame(iconic_taxon_name = c("Plantae", "Insecta", "Animalia", "Aves", "Fungi", "Mollusca", 6 | "Actinopterygii", "Arachnida", "Reptilia", "Mammalia", 7 | "Amphibia", "Chromista", "Protozoa"), 8 | Prey_Kingdom = c("Plantae", "Animalia", "Animalia", "Animalia", "Fungi", 9 | "Animalia", "Animalia", "Animalia", "Animalia", "Animalia", 10 | "Animalia", "Chromista", "Protozoa")) 11 | 12 | info <- sessionInfo() 13 | bioark <- ifelse(grepl("apple", info$platform), "/Volumes", "\\\\BioArk") 14 | setwd(paste0(bioark, "/HurlbertLab/Databases/iNaturalist/")) 15 | con <- DBI::dbConnect(RSQLite::SQLite(), dbname = "iNaturalist_s.db") 16 | db_list_tables(con) 17 | inat = tbl(con, "inat") 18 | 19 | # Takes ~20-30 minutes 20 | commNames = inat %>% 21 | distinct(scientific_name, common_name, iconic_taxon_name) %>% 22 | filter(common_name != "", 23 | !grepl("Unconfirmed", common_name)) %>% 24 | left_join(icon_kingdom, by = 'iconic_taxon_name') %>% 25 | select(-iconic_taxon_name) %>% 26 | distinct() %>% 27 | collect() 28 | 29 | 30 | diet = read.table('AvianDietDatabase.txt', header = T, sep = '\t', quote = '\"', fill = T) 31 | dietnames = distinct(diet, Prey_Kingdom, Prey_Phylum, Prey_Class, Prey_Order, Prey_Suborder, Prey_Family, Prey_Genus, Prey_Scientific_Name) 32 | 33 | phylumNames = dietnames %>% 34 | distinct(Prey_Kingdom, Prey_Phylum) %>% 35 | filter(Prey_Phylum != "", !is.na(Prey_Phylum)) %>% 36 | left_join(commNames, by = c('Prey_Phylum' = 'scientific_name', 'Prey_Kingdom' = 'Prey_Kingdom')) %>% 37 | filter(!is.na(common_name), 38 | common_name != 'Mosses') %>% # leaving Bryophyta = mosses, deleting duplicate Bryophyta = Mosses 39 | mutate(taxonomic_rank = 'Prey_Phylum') %>% 40 | rename(taxon = Prey_Phylum) %>% 41 | arrange(taxon) 42 | 43 | classNames = dietnames %>% 44 | distinct(Prey_Kingdom, Prey_Class) %>% 45 | filter(Prey_Class != "", !is.na(Prey_Class)) %>% 46 | left_join(commNames, by = c('Prey_Class' = 'scientific_name', 'Prey_Kingdom' = 'Prey_Kingdom')) %>% 47 | filter(!is.na(common_name)) %>% 48 | mutate(taxonomic_rank = 'Prey_Class') %>% 49 | rename(taxon = Prey_Class) %>% 50 | arrange(taxon) 51 | 52 | orderNames = dietnames %>% 53 | distinct(Prey_Kingdom, Prey_Order) %>% 54 | filter(Prey_Order != "", !is.na(Prey_Order)) %>% 55 | left_join(commNames, by = c('Prey_Order' = 'scientific_name', 'Prey_Kingdom' = 'Prey_Kingdom')) %>% 56 | filter(!is.na(common_name), 57 | !common_name %in% c('Even-toed Ungulates and Cetaceans', 'pinks, cacti, and allies')) %>% 58 | mutate(taxonomic_rank = 'Prey_Order') %>% 59 | rename(taxon = Prey_Order) %>% 60 | arrange(taxon) 61 | 62 | # Check names in this list for errors 63 | suborderNames = dietnames %>% 64 | distinct(Prey_Kingdom, Prey_Suborder) %>% 65 | filter(Prey_Suborder != "", !is.na(Prey_Suborder)) %>% 66 | left_join(commNames, by = c('Prey_Suborder' = 'scientific_name', 'Prey_Kingdom' = 'Prey_Kingdom')) %>% 67 | filter(!is.na(common_name), 68 | !common_name %in% c('')) %>% 69 | mutate(taxonomic_rank = 'Prey_Suborder') %>% 70 | rename(taxon = Prey_Suborder) %>% 71 | arrange(taxon) 72 | 73 | familyNames = dietnames %>% 74 | distinct(Prey_Kingdom, Prey_Family) %>% 75 | filter(Prey_Family != "", !is.na(Prey_Family)) %>% 76 | left_join(commNames, by = c('Prey_Family' = 'scientific_name', 'Prey_Kingdom' = 'Prey_Kingdom')) %>% 77 | filter(!is.na(common_name), 78 | !common_name %in% c('gourds, squashes, pumpkins, and allies', 79 | 'Cascade Beetle', 80 | 'ratanies', 81 | '', 82 | 'Tiphiid Flower Wasps')) %>% 83 | mutate(taxonomic_rank = 'Prey_Family') %>% 84 | rename(taxon = Prey_Family) %>% 85 | arrange(taxon) 86 | 87 | 88 | genusNames = dietnames %>% 89 | distinct(Prey_Kingdom, Prey_Genus) %>% 90 | filter(Prey_Genus != "", !is.na(Prey_Genus)) %>% 91 | left_join(commNames, by = c('Prey_Genus' = 'scientific_name', 'Prey_Kingdom' = 'Prey_Kingdom')) %>% 92 | filter(!is.na(common_name), 93 | !grepl(" sect. ", common_name), 94 | !common_name %in% c('Copperheads, Cottonmouths, and Cantils', 95 | 'Thimbleweeds, anemones, and windflowers', 96 | 'Cavity-nesting Honey Bees', 97 | 'Eurasian Water Voles', 98 | 'Water Melons', 99 | 'Sotol', 100 | 'Starapples', 101 | 'Cellophane-cuckoo Bees', 102 | 'Fig Shells', 103 | 'Witch hazels', 104 | 'Hog-nosed Snakes', 105 | 'Ilex oaks', 106 | 'Iris Mantises', 107 | 'needle-leaf junipers', 108 | 'Monkeyflowers','Annual or Dogday Cicadas', 109 | 'evening primroses, sundrops, and beeblossums', 110 | 'evening primroses, sundrops, and beeblossoms', 111 | 'Typical passionflowers', 112 | 'Eurasian, red, and tropical pines', 113 | 'hard pines', 114 | 'dominula-group Paper Wasps', 115 | 'Blackclocks', 116 | 'North American white oaks', 117 | 'high-latitude oaks', 118 | 'noseburn', 119 | 'Narrow-fronted Fiddler Crabs'), 120 | !(Prey_Kingdom == 'Animalia' & common_name == 'marram grasses'), 121 | !(Prey_Kingdom == 'Plantae' & common_name == 'Thread-waisted Sand Wasps')) %>% 122 | mutate(taxonomic_rank = 'Prey_Genus') %>% 123 | rename(taxon = Prey_Genus) %>% 124 | arrange(taxon) 125 | 126 | genusNames$common_name[genusNames$taxon == "Limnophila" & genusNames$Prey_Kingdom == "Plantae"] = "marshweed" 127 | genusNames$Prey_Kingdom[genusNames$taxon == "Oenanthe" & tolower(genusNames$iconic_taxon_name) == "water-dropworts"] = "Plantae" 128 | genusNames$Prey_Kingdom[genusNames$taxon == "Passerina" & tolower(genusNames$iconic_taxon_name) == "gonnas"] = "Plantae" 129 | 130 | # Currently there are 4 homonymns where the genus name is present in both Animalia and Plantae: 131 | # Limnophila, Oenanthe, Passerina, and Ammophila 132 | 133 | 134 | # Create entry for caterpillars to add to list: 135 | 136 | caterpillars = data.frame(taxon = 'Lepidoptera', rank = 'Prey_Order', commonName = 'caterpillars', 137 | Prey_Kingdom = 'Animalia', Prey_Stage = 'larva') 138 | 139 | commonNamesList = rbind(phylumNames, classNames, orderNames, suborderNames, familyNames, genusNames) %>% 140 | mutate(commonName = tolower(common_name), 141 | Prey_Stage = NA) %>% 142 | select(taxon, rank, commonName, Prey_Kingdom, Prey_Stage) %>% 143 | rbind(caterpillars) 144 | 145 | commonNamesList$Prey_Stage[commonNamesList$taxon == "Lepidoptera" & 146 | commonNamesList$commonName == "butterflies and moths"] = "adult" 147 | 148 | write.csv(commonNamesList, 'preyCommonNames.csv', row.names = F) 149 | 150 | -------------------------------------------------------------------------------- /training/AvianDietDatabase_AR.txt.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/training/AvianDietDatabase_AR.txt.txt -------------------------------------------------------------------------------- /training/AvianDietDatabase_RLC.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/training/AvianDietDatabase_RLC.txt -------------------------------------------------------------------------------- /training/training_dietdatabase.txt: -------------------------------------------------------------------------------- 1 | Common_Name Scientific_Name Subspecies Family Taxonomy Longitude_dd Latitude_dd Altitude_min_m Altitude_mean_m Altitude_max_m Location_Region Location_Specific Habitat_type Observation_Month_Begin Observation_Year_Begin Observation_Month_End Observation_Year_End Observation_Season Analysis_Number Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Inclusive_Prey_Taxon Prey_Name_ITIS_ID Prey_Name_Status Prey_Stage Prey_Part Prey_Common_Name Fraction_Diet Diet_Type Item_Sample_Size Bird_Sample_Size Sites Study_Type Notes Entered_By Source doi sex age_class within_study_data_source table_fig_number title lastname_author source_year journal 2 | Bald Eagle Haliaeetus leucocephalus NA Accipitridae eBird Clements Checklist v2019 NA NA NA NA NA Mexico Sonora shrubland NA 1987 NA 1989 summer 1 Animalia Chordata Teleostei Siluriformes Ictaluridae Ictalurus yes 163996 verified NA Catfish 0.178 Wt_or_Vol 118 NA 1 prey remains "habitat classified as Subtropical Sinaloan thornscrub; samples taken from 3 nests from nest bowls or under nests during/after nesting season; percentage mass is biomass, determined by average biomass for the prey species;" CS "Brown, B. T., W. C. Leibfried, T. R. Huels, and J. A. Olivera. 1991. Prey remains from Bald Eagle nests in Sonora, Mexico. Southwestern Naturalist 36:259-262." NA NA NA NA NA NA Brown 1991 NA 3 | -------------------------------------------------------------------------------- /training/training_dietdatabase_AD.txt: -------------------------------------------------------------------------------- 1 | Common_Name Scientific_Name Subspecies Family Taxonomy Longitude_dd Latitude_dd Altitude_min_m Altitude_mean_m Altitude_max_m Location_Region Location_Specific Habitat_type Observation_Month_Begin Observation_Year_Begin Observation_Month_End Observation_Year_End Observation_Season Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Unidentified Prey_Name_Status Prey_Stage Prey_Part Prey_Common_Name Fraction_Diet Diet_Type Item_Sample_Size Bird_Sample_Size Sites Study_Type Notes Entered_By Source 2 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Coleoptera no 109216 verified NA 0.3 Items 30 9 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 3 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta yes 99208 verified NA small winged insect 0.033 Items 30 9 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 4 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Hymenoptera Apocrita Formicidae no 154193 verified NA 0.033 Items 30 9 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 5 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Lepidoptera no 117232 verified larva NA 0.633 Items 30 9 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 6 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Coleoptera no 109216 verified NA 0.4444444 Occurrence 30 9 1 stomach contents JK "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 7 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta yes 99208 verified NA small winged insect 0.1111111 Occurrence 30 9 1 stomach contents JK "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 8 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Hymenoptera Apocrita Formicidae no 154193 verified NA 0.1111111 Occurrence 30 9 1 stomach contents JK "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 9 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Lepidoptera no 117232 verified larva NA 0.5555556 Occurrence 30 9 1 stomach contents JK "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 10 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta no 118840 verified NA 0.267 Items 15 3 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 11 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Diptera yes 118831 verified NA 0.067 Items 15 3 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 12 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Hymenoptera Apocrita Formicidae no 154193 verified NA 0.267 Items 15 3 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 13 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Hemiptera Heteroptera no 103358 verified NA 0.067 Items 15 3 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 14 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Hymenoptera Apocrita Ichneumonidae no 153360 verified NA 0.067 Items 15 3 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 15 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Lepidoptera no 117232 verified larva NA 0.267 Items 15 3 1 stomach contents Prey_Stage assumed to be larva due to foraging behavior of warblers AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 16 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Hemiptera Heteroptera Tingidae Tingis no 104395 verified NA 0.071 Items 14 1 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 17 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Insecta Lepidoptera no 117232 verified larva NA 0.643 Items 14 1 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 18 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Arachnida Coleoptera Polyphaga Chrysomelidae no 114509 verified NA 0.142 Items 14 1 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 19 | Blackburnian Warbler Setophaga fusca NA Parulidae eBird Clements Checklist v2018 NA NA NA NA NA Wisconsin Berlin forest 5 1874 5 1874 spring Animalia Arthropoda Arachnida Diptera Nematocera Tipulidae no 118840 verified NA 0.142 Items 14 1 1 stomach contents AHH "King, F. H. 1883. Economic relations of Wisconsin birds. Geology of Wisconsin 441-610." 20 | -------------------------------------------------------------------------------- /training/training_dietdatabase_JSW.txt: -------------------------------------------------------------------------------- 1 | Common_Name Scientific_Name Subspecies Family Taxonomy Longitude_dd Latitude_dd Altitude_min_m Altitude_mean_m Altitude_max_m Location_Region Location_Specific Habitat_type Observation_Month_Begin Observation_Year_Begin Observation_Month_End Observation_Year_End Observation_Season Analysis_Number Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Inclusive_Prey_Taxon Prey_Name_ITIS_ID Prey_Name_Status Prey_Stage Prey_Part Prey_Common_Name Fraction_Diet Diet_Type Item_Sample_Size Bird_Sample_Size Sites Study_Type Notes Entered_By Source 2 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Coleoptera Polyphaga Buprestidae yes (R fills in) (R fills in) 0.08 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 3 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Coleoptera Polyphaga Cerambycidae yes 0.04 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 4 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Coleoptera Polyphaga Chrysomelidae yes 0.04 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 5 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Coleoptera Polyphaga Curculionidae yes 0.08 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 6 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Coleoptera Polyphaga Scarabeidae yes 0.25 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 7 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Coleoptera Polyphaga Scolytidae yes 0.25 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 8 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Coleoptera Polyphaga no 0.41 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 9 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Diptera Brachycera Calliphoridae yes 0.03 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 10 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Diptera Brachycera Empididae yes 0.01 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 11 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Diptera Brachycera Muscidae yes 0.04 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 12 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Diptera Brachycera Rhagionidae yes 0.06 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 13 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Diptera Brachycera Sarcophagidae yes 0.01 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 14 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Diptera Brachycera Syrphidae yes 0.03 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 15 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Diptera Brachycera Tabanidae yes 0.06 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 16 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Diptera Brachycera Therevidae yes 0.01 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 17 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Diptera Brachycera no 0.75 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 18 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Hemiptera Heteroptera Corizidae yes 0.08 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 19 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Hemiptera Heteroptera Nabidae yes 0.16 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 20 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Hemiptera Heteroptera no 0.76 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 21 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Homoptera Heteroptera Cercopidae yes 0.4 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 22 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Homoptera Heteroptera Cicadellidae yes 0.3 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 23 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Homoptera Heteroptera no 0.3 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 24 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Hymenoptera Bombidae yes 0.01 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 25 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Hymenoptera Braconidae yes 0.02 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 26 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Hymenoptera Cephidae yes 0.01 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 27 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Hymenoptera Chalcidoidea yes 0.01 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 28 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Hymenoptera Formicidae yes 0.72 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 29 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Hymenoptera Ichneumonidae yes 0.02 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 30 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Hymenoptera Tenthredinidae yes 0.01 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 31 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Hymenoptera Vespidae yes 0.01 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 32 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Hymenoptera no 0.19 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 33 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Lepidoptera Geometridae yes 0.09 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 34 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Lepidoptera Nymphalidae yes 0.02 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 35 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Lepidoptera Tortricidae yes 0.28 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 36 | Western Wood-Pewee contopus sordidulus NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 1 Animalia Arthrapoda Insecta Lepidoptera no 0.61 Wt_or_Vol 334 69 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 37 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Coleoptera Polyphaga Cleridae yes 0.04 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 38 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Coleoptera Polyphaga Curculionidae yes 0.08 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 39 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Coleoptera Polyphaga Scarabeidae yes 0.04 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 40 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Coleoptera Polyphaga no 0.84 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 41 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Diptera Brachycera Empididae yes 0.03 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 42 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Diptera Brachycera Rhagionidae yes 0.13 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 43 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Diptera Brachycera Syrphidae yes 0.03 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 44 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Diptera Brachycera Tabanidae yes 0.1 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 45 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Diptera Brachycera no 0.71 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 46 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Hymenoptera Braconidae yes 0.06 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 47 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Hymenoptera Formicidae yes 0.37 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 48 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Hymenoptera Ichneumonidae yes 0.08 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 49 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Hymenoptera no 0.49 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 50 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Lepidoptera Tortricidae yes 0.62 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 51 | Pacific-slope flycatcher Empidonax difficilis NA Tyrannidae eBird Clements Checklist v2019 -105.1196 38.05489428 NA 2900 NA Colorado San Isabel National Forest forest NA 1965 NA 1966 summer 2 Animalia Arthrapoda Insecta Lepidoptera no 0.38 Wt_or_Vol 96 23 1 stomach contents JW "Beaver, D.L. and P.H. Baldwin. 1975. Ecological overlap and the problem of competition and sympatry in the Western and Hammond's Flycatchers. Condor 77: 1-13." 52 | -------------------------------------------------------------------------------- /training/training_dietdatabase_JSW_badnames.txt: -------------------------------------------------------------------------------- 1 | "level" "name" "condition" 2 | "Family" "Scarabeidae" "unmatched" 3 | "Family" "Scolytidae" "unmatched" 4 | "Family" "Corizidae" "unmatched" 5 | "Family" "Bombidae" "unmatched" 6 | "Family" "Chalcidoidea" "wrong rank; too low" 7 | -------------------------------------------------------------------------------- /training/training_dietdatabase_MAS.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hurlbertlab/dietdatabase/371b5a4e78fab7093edbcbdae6972901e53d7717/training/training_dietdatabase_MAS.txt -------------------------------------------------------------------------------- /training/training_dietdatabase_RLQ.txt: -------------------------------------------------------------------------------- 1 | Common_Name Scientific_Name Family Taxonomy Longitude_dd Latitude_dd Altitude_min_m Altitude_mean_m Altitude_max_m Location_Region Location_Specific Habitat_type Observation_Month_Begin Observation_Year_Begin Observation_Month_End Observation_Year_End Observation_Season Prey_Kingdom Prey_Phylum Prey_Class Prey_Order Prey_Suborder Prey_Family Prey_Genus Prey_Scientific_Name Unidentified Prey_Name_Status Prey_Stage Prey_Part Prey_Common_Name Fraction_Diet_By_Wt_or_Vol Fraction_Diet_By_Items Fraction_Occurrence Fraction_Diet_Unspecified Item_Sample_Size Bird_Sample_Size Sites Study_Type Notes Entered_By Source 2 | Yellow-billed cuckoo Coccyzus americanus Cuculidae "AOU 7th ed., 52nd supplement" NA NA NA NA NA Multiple Multiple 5 1897 10 1897 Summer Animalia Arthropoda Insecta Coleoptera verified NA NA NA 0.06 NA 109 999 stomach contents "Study performed in 20 states, district of columbia and canada. Study date assumed to be 1 year prior to publication." "Beal, F. E. L. 1898. Cuckoos and shrikes and their relation to agriculture. U.S. Dep. Agric. Biol. Surv. Bull. no. 9." 3 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Coleoptera Buprestidae verified Metallic Wood Boring Beetles NA 0.014 336 69 3 stomach contents RQ 4 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Coleoptera Cerambycidae NA 0.006 336 69 3 stomach contents RQ 5 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Coleoptera Chrysomelidae NA 0.006 336 69 3 stomach contents RQ 6 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Coleoptera Cleridae NA NA 336 69 3 stomach contents RQ 7 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Coleoptera Curculionidae NA 0.014 336 69 3 stomach contents RQ 8 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Coleoptera Scarabeidae NA 0.04 336 69 3 stomach contents RQ 9 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Coleoptera Scolytidae NA 0.016 336 69 3 stomach contents RQ 10 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Coleoptera yes NA 0.066 336 69 3 stomach contents RQ 11 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Diptera Calliphoridae NA 96 23 3 stomach contents RQ 12 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Diptera Empididae NA 96 23 3 stomach contents RQ 13 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Diptera Muscidae NA 96 23 3 stomach contents RQ 14 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Diptera Rhagionidae NA 96 23 3 stomach contents RQ 15 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Diptera Sarcophagidae NA 96 23 3 stomach contents RQ 16 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Diptera Syrphidae NA 96 23 3 stomach contents RQ 17 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Diptera Tabanidae NA 96 23 3 stomach contents RQ 18 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Diptera Therevidae NA 96 23 3 stomach contents RQ 19 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Diptera Tipulidae NA 96 23 3 stomach contents RQ 20 | Western Wood-Pewee Contopus sordidulus Tyrannidae American Ornithologists' Union 7th edition (incl. 54th suppl. -105.1037877 38.0404934 2900 Colorado San Isabel National Forest Deciduous Forest; coniferous forest 1965 1966 Summer Animalia Arthropoda Insecta Diptera yes NA 96 23 3 stomach contents RQ 21 | -------------------------------------------------------------------------------- /training/training_summary_notes.md: -------------------------------------------------------------------------------- 1 | # Training dataset notes 2 | 3 | ## Training Dataset 1. Beaver & Baldwin 1975 4 | 5 | ### Taxonomy 6 | 1. *Western flycatcher*. You should have found that Western Flycatcher (*Empidonax difficilis*) 7 | was split into two species, the Pacific-slope Flycatcher (which retained the scientific name 8 | *E. difficilis*) and the Cordilleran Flycatcher (*E. occidentalis*). How do you know which 9 | of these two is represented by the study? By examining the range maps 10 | [here](http://avibase.bsc-eoc.org/species.jsp?lang=EN&avibaseid=44A2028364A252A6&sec=ebird) and 11 | [here](http://avibase.bsc-eoc.org/species.jsp?lang=EN&avibaseid=6FD04659A3D0CAC8&sec=ebird). 12 | The study was done in Colorado, so clearly they must have been examining the Cordilleran Flycatcher. 13 | This means both the common name and the scientific name must be altered in the Diet Database. 14 | 15 | 2. *Homoptera*. Refers to leafhoppers, treehoppers, and cicadas and was historically considered 16 | an Order, but is now considered to be a suborder of Hemiptera by ITIS. Furthermore, its new 17 | name is [Auchenorrhynca](http://resolver.globalnames.org/name_resolvers/ffrsyur980io). 18 | 19 | 3. *Bombidae*. Bumblebees, used to be considered a full family, but now is considered a 20 | subfamily Bombinae within the family Apidae. 21 | 22 | 4. *Choristoneura fumiferana*. The spruce budworm in the Lepidoptera family Tortricidae is 23 | now known as *Archips fumiferana* according to [ITIS](http://resolver.globalnames.org/name_resolvers/kku1hn1kuimh). 24 | 25 | ### Diet fractions 26 | 5. *Source*. Both Tables 4 and 5 include quantitative information on diet, but in Table 4, the data 27 | reflect fraction of the diet by number of items, while in Table 5 the data reflect fraction by weight. 28 | This means that the Orders should each have two diet fraction entries within each row: 29 | Fraction_Diet_By_Items from Table 4 and Fraction_Diet_By_Wt_or_Vol from Table 5. 30 | 31 | 6. *Table 4*. Diet should be recorded to the finest taxonomic resolution possible. In most cases 32 | here this is Family, but the values provided are percentages of the Order that family falls in 33 | rather than percentages of the diet overall. This means you must multiply the Order % by the Family %. 34 | For example, Buprestidae makes up 8% of Coleoptera, which make up 16% of the diet, so Buprestidae 35 | makes up 1.28% (or a diet fraction of 0.0128, you can round to 0.013) of the overall diet, etc. 36 | 37 | ### Other 38 | 7. *Longitude and Latitude*. The study reports a location using "Township and Range" coordinates, 39 | T.23S., R.69W. These can be converted to longitude and latitude through websites like this 40 | [one](http://www.earthpoint.us/TownshipsSearchByDescription.aspx). 41 | 42 | 8. *Location_Specific*. Either "San Isabel National Forest" or "Wet Mountains, San Isabel National 43 | Forest" would be fine here. 44 | 45 | 9. *Habitat_type*. This study mentions collection in both deciduous and coniferous forest. We can 46 | list them both, separated by a ";". Remember to stick to just these few standardized habitat names: 47 | +deciduous forest 48 | +coniferous forest 49 | +woodland 50 | +scrubland 51 | +grassland 52 | +desert 53 | +wetland 54 | +agriculture 55 | +urban 56 | 57 | 58 | ## Training Dataset 2. Allaire & Fisher 1975 59 | 60 | ### Taxonomy 61 | 1. *Aimophila aestivalis*. The scientific name for Bachman's Sparrow was changed to 62 | *Peucaea aestivalis*. 63 | 64 | 2. *Genus sp.*. When an entry is listed as, for example, "Digitaria sp.", go ahead 65 | and enter the Prey_Genus as "Digitaria", and set the Unidentified field to "no". This 66 | implies that the data represents all members of this genus. 67 | 68 | 3. *Graminae*. This is an old family name for Grasses--the current name is "Poaceae". 69 | Also, two species labeled "Graminae sp. 1" and "Graminae sp. 2" are listed, however we have 70 | no way of linking these names to an existing taxonomic entity. As such, we will simply 71 | create an entry for Prey_Family "Graminae" where Unidentified is "yes" and include the 72 | sum of both of these species in a single entry. (E.g., for Field Sparrow in winter, the 73 | Fraction_Diet_By_Items would be (0.46 + 0.14) times the fraction of all prey that are seeds; see below.) 74 | 75 | 4. *Compositae*. The new name of this plant family is "Asteraceae". 76 | 77 | 5. *Leguminosae*. The new name of this plant family is "Fabaceae". 78 | 79 | 6. *Triodia*. This name has no entry in the Global Names Resolver from the ITIS database, 80 | but it is listed under NCBI, so we will consider this a good name. 81 | 82 | 7. *Homoptera*. See Training Dataset 1 notes. 83 | 84 | 8. *Arachnoides / Araneae*. Aranaea is a good Order, but the Global Names Resolver indicates 85 | that it should be considered in the Class Arachnida, not Arachnoides. 86 | 87 | 9. *Isoptera*. The termites used to be considered their own order, but now are considered 88 | to be within the order Blattodea which includes the cockroaches. But since termites and 89 | cockroaches are pretty distinctive, we still want to separate them. In this case, we'll 90 | keep Isoptera as the Prey_Suborder. 91 | 92 | ### Diet data 93 | 10. *Seeds*. Be sure to specify "seed" in the Prey_Part field when entering all of the 94 | data from Table 1. Otherwise, it would be unclear whether the birds were eating 95 | fruits, or leaves or other plant parts. Prey_Part is especially important for characterizing 96 | plant diet items in general. 97 | 98 | 11. *Calculating the fraction*. As in the training example, to get an estimate of the fraction 99 | of a given diet item in the overall diet, you need to multiply the reported % by the fraction 100 | of all diet items that are seeds (for Table 1) or arthropods (for Table 2). For example, 101 | Digitaria makes up 92.2% of the seed diet for Field Sparrow in the summer, and the observed 102 | diet consisted of 220 seeds and 12 arthropods. In this example, the Item_Sample_Size will be 232 103 | total diet items. So, Fraction_Diet_By_Items = 92.2 * (220 / 220 + 12) = 87.4% or 0.874. 104 | Because data were not simply transcribed from a table, you should make a comment in the Notes 105 | field like "values provided by source are for % of seed diet and % of insect diet; % of total 106 | diet calculated based on relative abundance of seeds and insects." 107 | 108 | ### Other 109 | 12. *Bird Sample Size*. This is given in one of the bottom rows of each table. Note that in 110 | a given season, they collected (i.e. shot) birds, sorted their stomach contents, and then 111 | broke the data down into these two tables. The total number of birds examined should be 112 | the same in each table for a given species and season. This helps clue us into the fact 113 | that the listing of 44 Bachman's Sparrows being examined for seeds in the winter must be a 114 | typo, since a) only 4 stomachs had seeds, and b) only 4 stomachs were examined for insects. 115 | Need to add a note to this effect in the Notes field. 116 | 117 | 13. *Longitude and Latitude*. Not provided, but pasting "Nacogdoches County, Texas lat long" 118 | into Google provides estimates that can be used. --------------------------------------------------------------------------------