Known Issues

Known Issues

The following are known issues for the October 2021 release.

User Interface Issues

1. Only NDVI and ET are available as raster views

    The requirements for visualization of satellite data products developed by the OpenET partners and working group members only included NDVI and ET for the raster view. Additional data layers will be added to the raster layer viewer in the future. 

    2. Model selection for the monthly timeseries graphs is reset with each new query

      Model selection for the monthly timeseries graphs is reset with each new query to avoid overloading the server with unnecessary queries and to increase response time for individual queries. Model selection may be changed in the future to be “sticky” based on query response times and user feedback. Queries for multiple models for large numbers of fields should primarily be handled through the API in the future.

      3. Only monthly data are available in OpenET Data Explorer’s field view

      At this time, only monthly data are available through the Data Explorer field view, while both monthly and daily data are available in raster view. The daily data volume is so large that it precludes inclusion in the geodatabase. However, summaries of daily data for regions of interest are available through the ‘Draw Custom Area’ tool, and will also be available through the Custom Reporting Tool when it is launched in 2024.

      General Data Issues

      1. EToF values do not agree well, even when ET values do agree across the ensemble

        The fraction of reference ET (EToF) is calculated as the ratio of ET to ETo (grass reference ET). When ETo is low, small variations in estimated ET will translate to large differences in EToF. In general, EToF values from different models should agree well during the growing season. Large differences in EToF during the winter are expected, even when there appears to be generally good agreement among the ET values. 

        2. The range of the model ensemble (the difference between the lowest and highest model) appears large for fallow fields, pasture, and other polygons with low ET.

            The OpenET team has observed greater disagreement across the ensemble of models for low ET conditions that include fallow fields, partially irrigated pasture lands, and other fields with low ET. The OpenET team is currently working to explain and resolve these differences. Because the y-axis in the graphing windows scales based on the maximum ET values for a given field, fields with lower ET values will have a smaller range in the y-axis and, as a result, smaller differences in ET among the models will be exaggerated in the graphs relative to fields with higher monthly and annual ET values.

            3. Crop type information for the current year is incorrect, or crop type information for California appears to be incorrect for years other than 2016 and 2018.

            Crop type information for the current year is incorrect, or crop type information for California appears to be incorrect for years other than 2016 and 2018. 

            Crop type information for the current year is derived from the previous year, and is provided to assist with interpretation of ET values. For many of the ET models used in OpenET, mis-classified crop type does not impact model performance.

            Crop type data for California for 2016, 2018 and 2020 are based on the CA Department of Water Resources (DWR) Land Cover database developed by LandIQ. Data for annual crops in California for all other years are from the USDA Cropland Data Layer which has lower classification accuracies for many of the specialty crops grown in California. Crop type information will be updated for California as additional data updates are made publically available by CA DWR. Outside of California, all crop type information is calculated from the USDA Cropland Data Layer

            Any errors in crop type do not affect the accuracy of the ensemble ET value, or the models that use thermal infrared data as an input for calculation of ET.

            Crop type information is not used as an input for five of the models on the OpenET platform (ALEXI/DisALEXI, eeMETRIC, geeSEBAL, PT-JPL, SSEBop). The sixth model, SIMS, model does use crop type information to separate annual and perennial crops, and to adjust model parameters for stomatal resistance and maximum crop height for perennial crops. However, errors in crop type classification for annual crops don’t affect the SIMS ET calculations, and the crop-specific adjustments for perennials usually have a small influence on SIMS ET relative to other inputs. In cases where crop type errors do have an effect on SIMS ET data (e.g., misclassification of a perennial crop as an annual crop), SIMS is typically flagged as an outlier and removed from the ensemble prior to calculation of the ensemble ET value.

            4. Field boundaries / data appear to be missing

            OpenET uses publicly available field boundary datasets. In some locations, these datasets are incomplete or missing large numbers of fields. The OpenET team is currently working with partners to improve and update these datasets. The Data Explorer ‘Raster View’ currently allows users to define any location of interest up to 1000 hectares (2,470 acres) and request data for the user-defined region.  The API allows users to define locations of interest and upload boundaries (shapefile, geojson, etc.) to obtain data summaries for those regions.

            5. ET estimates do not agree well over open water bodies

            This is a known issue and the OpenET team is currently collaborating with other scientists to implement a method specifically for calculation of open water evaporation from satellite data and meteorological data. OpenET data over open water currently provided through the OpenET Data Explorer are provisional and likely to change in future updates.

            6. There are some locations where ET is zero in desert environments

            Evaluation of initial results from application of the Mean Absolute Deviation (MAD) approach for outlier detection highlighted a limitation in sparse vegetation desert environments during the warm season, where ET rates are very low. In these cases, the majority of models within the ensemble have a tendency to consistently estimate ET values of zero, while one or two other models provide estimates of low, but non-zero ET values. Requiring a minimum of four models allows non-zero values to be included in the ensemble average in most cases.  However, in some locations, at least four models are providing ET estimates of zero.


            7. There are some locations where data is missing for a month for one or more models.

            There are some time periods and locations where data is missing for a month for one or more models. This is usually due to the lack of high quality satellite observations due to persistent cloud cover or snow cover during the missing month. Ongoing work by the OpenET team is focused on addressing these data gaps through data fusion techniques and increased use of data from Sentinel-2. Moving forward, we plan to reduce these gaps by using more satellite observations. 

            8. The OpenET Ensemble Value appears low for small agricultural regions in very arid environments

            From the limited number of cropland in-situ flux stations located in very arid environments, it is evident that some models have a systematic low bias for smaller agricultural areas in very arid regions, and the MAD outlier filtering approach does not filter outliers as desired due to the large range is model estimates. This may result in a low bias in the ensemble average. These locations are often indicated by fields with a wide range of ET values across the ensemble of ET models. Over the coming months, the OpenET team will continue to conduct additional research in these more challenging settings and develop a Best Practices Manual that will provide more region and application specific guidance. The ensemble value is likely to evolve in the coming year as the team conducts additional research and designs more region-specific approaches for calculation of the ensemble ET value. We encourage users to rely upon their knowledge of local conditions in applying the ensemble ET value, or selecting a single model or subset of models for use in their application.

            Model-specific Data Issues

            1. ALEXI/DisALEXI

            DisALEXI was recently ported to Google Earth Engine as part of the OpenET framework, and refinement is ongoing. A few known issues in the current version of DisALEXI are noted here, which the team is working to resolve.

            Underestimation in semi-arid regions

            ET from isolated irrigated areas in semi-arid regions may be underestimated in some cases.  This can result from misregistration of GOES-based ALEXI data with respect to Landsat, particularly in areas of strongly varying terrain, and from limitations of the modified Priestley-Taylor parameterization of transpiration used in the current TSEB formulation in capturing advection. A Penman-Monteith based version of TSEB is under evaluation using Google Earth Engine, and will be implemented in future collections pending satisfactory results. 

            Topographic biases

            DisALEXI exhibits biases in areas of strong topographic slope, with high ET bias preferentially on shaded hillslopes.  Future refinements will include the slope and aspect corrections to solar radiation load used in eeMETRIC.  This will improve hillslope ET estimates, as well as disaggregated fluxes from flat surfaces in ALEXI pixels containing high topographic variability which are effectively biased low in compensation.

            CFSR meteorological inputs and insolation monthly scaling flux

            To be consistent with continental-scale ALEXI execution, DisALEXI is currently using meteorological inputs from the Climate Forecast System Reanalysis (CFSR) – a coarse-resolution global dataset. Differences between the meteorological forcings (insolation in particular) will lead to differences in model output at the daily timestep – for example, DisALEXI may be significantly lower than other models on Landsat overpass days when CFSR has differences in cloud cover.

            Following the ALEXI procedure, interpolation to monthly ET for DisALEXI uses daily insolation rather than gridMET ETo as the scaling flux.  This will also introduce differences in comparison with other monthly datasets from other models, although these differences are not large in most regions of the modeling domain.

            In future work, DisALEXI will migrate to the common insolation and ETo scaling flux datasets used by the other OpenET models.  This will help to reduce non-model related differences with respect to the ensemble. 

            Coastal areas

            A high bias is evident in DisALEXI monthly fluxes along the Pacific Coast and the Great Lakes.  This is related to enhancement in ALEXI 4km-resolution ET in coastal pixels extending into the ocean, serving to similarly enhance disaggregated 30m fluxes within that and surrounding ALEXI pixels. These high coastal pixels are excluded as outliers in the ensemble product and do not affect monthly ensemble averages.  The high bias will be corrected in future work.

            2. eeMETRIC

            Some uncertainty in monthly estimates by eeMETRIC, as well as with other methods, is caused by infrequent clear satellite images during cloudy periods or in locations that are prone to clouds. Infrequent images may exhibit wetness or dryness associated with timing of precipitation events that is not representative of a monthly period.  During winter periods, thermal contrasts within an image may be too small for eeMETRIC to distinguish differences caused by ET. 

            Evaporation from open water is estimated by eeMETRIC using an aerodynamic equation rather than using a surface energy balance. This is done due to large uncertainties associated with heat storage fluxes into water bodies. However, there is uncertainty associated with the estimation of vapor pressure over water that is required in a full aerodynamic method, as well as whether wind speeds during the satellite overpass time are representative of those over a month. Estimation of evaporation from open water will progress in eeMETRIC over the next year.

            3. PT-JPL

            For croplands, wetlands, and riparian areas in arid and semiarid environments where advection is prevalent, a PT alpha adjustment layer was developed based on the ratio of ASCE reference ET and Priestley-Taylor ETw, following principles of the complementary relationship of evaporation (Kahler and Brutsaert, 2006; Szilagyi, 2007; Huntington et al., 2011). ASCE reference ET and ETw estimates were developed using the gridMET dataset. The alpha adjustment layer was calculated as the ratio of the growing average bias corrected ASCE reference ET to the growing season average ETw. Growing seasons were defined for each gridMET grid cell based on cumulative growing degree days of daily average air temperature from January 1 and killing frost thresholds of 300 C and -2 C, respectively. Adjustment factors were limited to a minimum of 0.79 and maximum of 2 to account for uncertainty in weather data and model performance in locations with extreme climate and weather (e.g. Death Valley, coastal regions). Adjusted alpha values range from 1 to 2.5, and fall within the range of alpha values previously reported across arid and humid settings (Priestley and Taylor, 1972; McAneney and Itier, 1996; Weiß and Menzel, 2008; Yang et al., 2009; Tabari and Talaee, 2011). Application of adjusted alpha values were limited to cropland, wetland, and riparian areas as defined by the USDA cropland data layer (CDL).

            Results from Phase II of the OpenET Accuracy Assessment and Intercomparison study indicated that application of the PT alpha adjustment layer increased overall accuracy of PT-JPL for cropland, wetland, and riparian areas. In cases where land cover information is incorrect, the correction may not have been applied, resulting in a low bias in estimates from PT-JPL. Future enhancements to PT-JPL will include the slope and aspect corrections to short and longwave radiation and LST inputs, similar to eeMETRIC, and further improvements to the PT alpha adjustment layer.

            4. SIMS

            The SIMS model generally performs well over irrigated agricultural areas and shows strong agreement with the ground-based ET datasets used in the OpenET Intercomparison and Accuracy Assessment. However, a positive bias in SIMS ET estimates for rainfed croplands, deficit irrigated fields, and fallow agricultural lands is a known issue. The SIMS model is primarily responsive to satellite measurements of green vegetative cover and assumes well-watered conditions. As a result, SIMS can overestimate ET from partially irrigated or deficit irrigated fields and long-term fallow areas, especially when grasses or weeds are present but not irrigated. Following the Phase I accuracy assessment, the team made improvements to the SIMS equations to address over-estimation of ET during periods with sparse vegetative cover by adjusting the Kcb minimum values. This improvement reduced the modest overall positive bias observed in the SIMS data at monthly and annual timesteps. The team has also implemented a gridded soil water balance model on Earth Engine to improve estimation of evaporation from bare soil and identify periods when available soil water in the root zone is likely to serve as a constraint on ET for non-irrigated lands.  

            SIMS was developed to support ET estimation and irrigation management over agricultural lands. For the purposes of providing continuous ET estimates across the western U.S., a simple reflectance-based model was implemented for non-agricultural land cover types. This approach was based on a linear transformation of NDVI, and did not incorporate the full density-coefficient approach used for agricultural lands. As a result, it frequently produced ET estimates with a very high positive bias for non-agricultural land cover types and these data were excluded from inclusion in the OpenET ensemble value. The data is included in the SIMS archive and accessible via Data Explorer queries in ‘raster view’ or the OpenET API. For non-agricultural areas, SIMS ET data should be interpreted as an upper limit on ET and a measure of potential ET that accounts for the vegetation density and condition. Future work by the SIMS team will extend the density coefficient approach used by SIMS to non-agricultural land cover types, and incorporate the soil water balance model to account for soil water limitations on ET. 

            5. geeSEBAL

            geeSEBAL performance is dependent on (i) the domain area of application, including topography and climate conditions, and (ii) the endmembers selection for automated calibration. Overall, some underestimation is observed in agricultural areas (especially in annual crops) and in arid and temperate climates with dry summers. Most of the uncertainties are related to the endmembers selection for automated calibration.The team is currently working to improve these limitations to increase model accuracy at multiple climate conditions and complex topographic landscapes. Some of the limitations are discussed below.

            Arid areas and climates with dry summers

            In desert areas and arid or semiarid climates, including areas with dry summers, geeSEBAL tends to yield lower ET estimates, producing ET estimates of zero or nearly zero over sparse vegetation areas. On the other hand, some vegetated areas can yield higher values, overestimating evapotranspiration in these conditions. This is possibly caused by the elevated temperatures in some land cover classes, such as bare soils or sparsely vegetated areas.

            Endmembers selection for automated calibration

            The geeSEBAL team is currently working on improving the endmembers selection algorithm to provide more accurate ET estimations, especially in arid climates and sparsely vegetated areas. The selection of accurate hot and cold endmembers will determine the accuracy of sensible heat estimates and therefore latent heat. Searching accurate endmembers over large and complex areas, with multiple climate and land cover conditions is challenging, so our solution is based on multiple strategies, including search algorithms and additional filters (available energy, slope, albedo, soil type, NDVI and LST) to select the endmembers over homogeneous areas and to accurately estimate sensible and latent heat fluxes.

            Water bodies

            The geeSEBAL team members are currently conducting an assessment and validation of evaporation estimates over water bodies, to investigate and characterize model uncertainties in these areas. Future improvements to geeSEBAL will include separate solutions for open water evaporation, including a better representation of the heat transferred to the water column. 

            6. SSEBop

            The following known issues, or causes of SSEBop ET model error, have been identified for user interpretation and awareness. Future improvements will address any known biases and utilize improved input data availability and data quality where available.

            Cloud contaminants and mountain shadows

            The SSEBop model uses Landsat Collection 2 (C2) source QA_PIXEL flags for masking unusable pixels (cloud, cloud shadow, snow, etc.) during image processing. SSEBop does not currently apply additional cloud screening or cloud/shadow-buffering techniques during processing. In some instances, model estimates may result in outlier ET values due to cloud mask errors. Similarly, valid pixels for mountain shadows in complex terrain have potential to create abnormal (low) land surface temperature (ST) values, leading to ET overestimation bias on shaded hillslopes. 

            Missing ST auxiliary data  

            Both reflective and thermal Landsat bands are required for the successful processing of SSEBop ET. Therefore, OLI-only or TIRS-only Landsat 8 or Landsat 9 scenes (e.g., “Tier 2”) cannot be used. Please note SSEBop ET may contain NoData pixels in images due to missing emissivity auxiliary data required for C2 ST processing. For additional information please visit Landsat C2 Level-2 ST Data Caveats.