├── CropSight-US └── CropSight - Sampling for Each Agricultural Statistics Districts (ASD).ipynb ├── Example └── CropSight.ipynb ├── README.md ├── code ├── Segment Anything.ipynb └── crop_type.ipynb └── src ├── Brazil_Map_S4.png ├── CropBoundary.png ├── CropGSV dataset.png ├── CropSight Examples.png ├── CropSight Flowchart Fig2.png ├── CropSight Flowchart.png ├── CropSight-US-Crop-Types.jpg ├── CropSight-US-Flowchart.png ├── Fig18.png ├── GSV_metadata.jpg ├── New Microsoft PowerPoint Presentation.gif ├── Panoramic clipping.gif ├── QDWxWb7c1sVcLeoquYPTHw_41.54170024691091_-88.29999867944885_2021_8_358.7471008300781.jpg ├── QDWxWb7c1sVcLeoquYPTHw_41.54170024691091_-88.29999867944885_2021_8_358.7471008300781_pano.jpg ├── SAM.png ├── VitResnet.jpg └── Zoom-in.gif /README.md: -------------------------------------------------------------------------------- 1 | # [CropSight: towards a large-scale operational framework for object-based crop type ground truth retrieval using street view and PlanetScope satellite imagery](https://www.sciencedirect.com/science/article/pii/S0924271624002922) 2 | 3 | ## Introduction 4 | 5 | Collecting accurate ground truth data of crop types is a crucial challenge for agricultural research and development. The CropSight Framework is an open-source toolkit designed to automate the retrieval of object-based crop type information from massive Google Street View (GSV) images. With its scalable and efficient features, CropSight enables the automatic identification of GSV images and boundary delineation to generate in-situ object-based crop-type labels over large areas. 6 | 7 | ### Key Components 8 | - **Large-Scale Operational Cropland Field-View Imagery Collection Method**: Systematically acquires representative geotagged cropland field-view images. 9 | - **Uncertainty-Aware Crop Type Image Classification Model (UncertainFusionNet)**: Retrieves high-quality crop type labels with quantified uncertainty. 10 | - **Cropland Boundary Delineation Model (SAM)**: Delineates cropland boundaries using PlanetScope satellite imagery. 11 | 12 | ## Workflow 13 | 14 |

15 | 16 |
17 | Figure 1: CropSight Flowchart. 18 |

19 | 20 | ## Dataset 21 | - ### UncertainFusionNet 22 |

23 | 24 |
25 | Figure 2: Crop type ground-level view dataset (CropGSV) used to train UncertainFusionNet. 26 |

27 | 28 | - ### SAM 29 |

30 | 31 |
32 | Figure 3: Cropland boundary ground-truth dataset (CropBoundary) used to fine-tune SAM. 33 |

34 | 35 | ## Application 36 | 37 | Using the CropSight framework, we collected crop type ground truth data from Google Street View and PlanetScope satellite imagery. Below are examples of the application of CropSight in the US and Brazil. 38 | 39 | - ### Example 1: Brazil 40 | 41 |

42 | 43 |
44 | Figure 4: Object-based crop type ground truth map produced by CropSight using the latest images (2023) in Brazil. Crop type labels are overlaid on Google Earth imagery. The accuracy of crop type classification and boundary delineation is assessed by randomly sampling and comparing against visually interpreted GSV-based ground truth data. 45 |

46 | 47 | - ### Example 2: United States 48 | 49 |

50 | 51 |
52 | Figure 5: Object-based crop type ground truth maps produced by CropSight using the latest images (2023). These maps represent four distinct study areas in the United States (A-D). (a) Overlay of crop type labels on Google Maps. (b) Overlay of crop type labels on off-season PlanetScope images. 53 |

54 | 55 | 56 | ## Example of Retrieving One Ground Truth 57 | 58 | To see an example of how to retrieve one ground truth using the CropSight framework, refer to the [CropSight.ipynb](https://colab.research.google.com/drive/1yoTC0MrmTVOrDZNF7A7rNcK-XbthJ1Ub?usp=drive_link). 59 | 60 | ## CropSight-US: A National-Scale Object-based Crop Type Ground Truth Dataset 61 | 62 | CropSight-US is an annual, object-based crop type ground truth dataset covering the contiguous United States (CONUS) from 2013 to 2023. Based on the CropSight workflow (Liu et al., 2024), it expands sample generation from specific sites to nationwide coverage, labeling 17 distinct crop types. The dataset integrates Google Street View imagery for crop type identification and Sentinel-2 imagery for field boundary delineation, addressing the challenge of large-scale ground truth data collection. To our knowledge, CropSight-US is the first nationwide, object-based crop type dataset derived from street view imagery, offering broad spatial and crop-type coverage. 63 | 64 | CropSight-US is in its final stages of preparation and will be released soon as an open-source dataset. 65 | 66 |

67 | 68 |
69 | Figure 6: CropSight-US ground-truthing framework demonstrating the steps necessary to generate the CropSight-US products across CONUS for object-based crop type ground truth building on the CropSight by Liu et al. (2024). 70 |

71 | 72 | We constructed a field-level crop type ground truth dataset using an object-based framework with 17 major crop types sampled from the GSV metadata pool. Each record includes predicted crop type, confidence level (from CONUS-UncertainFusionNet), cropland boundary, and image timestamp (year, month). For each crop, we used CSB data to compute the average number of fields per ASD and assigned ASDs to quantiles (Q1–Q4). GSV metadata were aggregated per crop-ASD pair, excluding crops with <2% total metadata. If an ASD had ≤ average GSV entries, all were used. For ASDs above average, extra samples were drawn at 0.2x, 0.4x, 0.6x, and 0.8x the excess count for Q1–Q4, respectively. Sampling was stratified by ASD-level irrigation:rainfed ratios, with targets decomposed accordingly. Within each ASD, a spatially adaptive fishnet approach ensured spatially representative sampling per crop. More information about sampling is documented at [CropSight-ASD-GSV-Sampling.ipynb](https://colab.research.google.com/drive/1lBX9MaaueqojQ3JpbS0WaNvNqeS7R_UI?usp=sharing) 73 | 74 |

75 | 76 |
77 | Figure 7: Samples of the reference dataset showcasing field-view images of 17 crop types included in CropSight-US. 78 |

79 | 80 | 81 | ## Author 82 | Yin Liu (yinl3@illinois.edu) 83 | 84 | Zhijie Zhou (zhijiez2@illinois.edu) 85 | 86 | Chunyuan Diao (chunyuan@illinois.edu) 87 | 88 | [Remote Sensing Space-Time Innovation Lab](https://diaorssilab.web.illinois.edu/) 89 | 90 | Department of Geography & GIScience, University of Illinois at Urbana-Champaign 91 | 92 | 93 | ## Acknowledgement 94 | This project is supported by the National Science Foundation’s Office of Advanced Cyberinfrastructure under grant 2048068. 95 | 96 | ## Citation 97 | If you use this work in any way, please mention this citation: 98 | ```markdown 99 | @article 100 | {Title: CropSight: towards a large-scale operational framework for object-based crop type ground truth retrieval using street view and PlanetScope satellite imagery, 101 | Authors: Liu, Yin and Diao, Chunyuan and Mei, Weiye and Zhang, Chishan, 102 | Publication: ISPRS Journal of Photogrammetry and Remote Sensing, 103 | Year: 2024, 104 | Volume:216 105 | Page: 66-89, 106 | DOI: 10.1016/j.isprsjprs.2024.07.025} 107 | -------------------------------------------------------------------------------- /src/Brazil_Map_S4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/Brazil_Map_S4.png -------------------------------------------------------------------------------- /src/CropBoundary.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/CropBoundary.png -------------------------------------------------------------------------------- /src/CropGSV dataset.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/CropGSV dataset.png -------------------------------------------------------------------------------- /src/CropSight Examples.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/CropSight Examples.png -------------------------------------------------------------------------------- /src/CropSight Flowchart Fig2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/CropSight Flowchart Fig2.png -------------------------------------------------------------------------------- /src/CropSight Flowchart.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/CropSight Flowchart.png -------------------------------------------------------------------------------- /src/CropSight-US-Crop-Types.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/CropSight-US-Crop-Types.jpg -------------------------------------------------------------------------------- /src/CropSight-US-Flowchart.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/CropSight-US-Flowchart.png -------------------------------------------------------------------------------- /src/Fig18.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/Fig18.png -------------------------------------------------------------------------------- /src/GSV_metadata.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/GSV_metadata.jpg -------------------------------------------------------------------------------- /src/New Microsoft PowerPoint Presentation.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/New Microsoft PowerPoint Presentation.gif -------------------------------------------------------------------------------- /src/Panoramic clipping.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/Panoramic clipping.gif -------------------------------------------------------------------------------- /src/QDWxWb7c1sVcLeoquYPTHw_41.54170024691091_-88.29999867944885_2021_8_358.7471008300781.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/QDWxWb7c1sVcLeoquYPTHw_41.54170024691091_-88.29999867944885_2021_8_358.7471008300781.jpg -------------------------------------------------------------------------------- /src/QDWxWb7c1sVcLeoquYPTHw_41.54170024691091_-88.29999867944885_2021_8_358.7471008300781_pano.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/QDWxWb7c1sVcLeoquYPTHw_41.54170024691091_-88.29999867944885_2021_8_358.7471008300781_pano.jpg -------------------------------------------------------------------------------- /src/SAM.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/SAM.png -------------------------------------------------------------------------------- /src/VitResnet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/VitResnet.jpg -------------------------------------------------------------------------------- /src/Zoom-in.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rssiuiuc/CropSight/94d7ece06c9b7c8bc39433ed925e5c7cabacc344/src/Zoom-in.gif --------------------------------------------------------------------------------