Get the latest tech news
A journey of optimization of cloud-based geospatial data processing
A journey of optimization of cloud-based geospatial data processing.
The rapid growth of Earth observation data in cloud storage, which will continue to grow exponentially, powered by falling rocket launch prices by companies like SpaceX, has pushed us to think of how we access and analyze satellite imagery. Filter pushdown for spatial and temporal fields Efficient compression of repeated values Reduced I/O through column pruning Fast parallel processing capabilities with the right parquet reading libraries Pure Python or Rust implementations of operations like rasterio.mask Adding more data sources like USGS Landsat and others LRU or other cache for repeated same tile queries Reducing memory usage Benchmark against Xarray and Dask workloads Test on multiple polygons across the world for 1 year date range
Or read this on Hacker News
