Skip to content

ENH: allow users to clear cache associated with vsicurl#605

Open
matteomorlack wants to merge 4 commits into
geopandas:mainfrom
matteomorlack:ENH-vsicurl-clear-cache
Open

ENH: allow users to clear cache associated with vsicurl#605
matteomorlack wants to merge 4 commits into
geopandas:mainfrom
matteomorlack:ENH-vsicurl-clear-cache

Conversation

@matteomorlack
Copy link
Copy Markdown

This PR proposes the addition of methods to allow users to clear the /vsicurl/ internal GDAL cache.

Implemented methods support both a full cleanup and a cleanup by prefix, using the provided methods VSICurlClearCache and VSICurlPartialClearCache

Basic tests are included to verify that the methods do not raise exceptions. The behavior should be tested more extensively against different GDAL builds; if you have a better testing approach, let me know and I will add it.

To test the functionaly one can:

import os
from pyogrio import read_dataframe, vsi_curl_clear_cache, set_gdal_config_options

os.environ["AWS_NO_SIGN_REQUEST"] = "YES"

set_gdal_config_options({"CPL_CURL_VERBOSE": True})

# observe from debug that a curl request is issued
read_dataframe("s3://mcsc-geotide-geojson-files/geojson_files/bayarea.geojson")

# observe from debug that NO curl request is issued
read_dataframe("s3://mcsc-geotide-geojson-files/geojson_files/bayarea.geojson")

vsi_curl_clear_cache("s3://") 

# observe from debug that a new curl request is issued, given cache has been cleared
read_dataframe("s3://mcsc-geotide-geojson-files/geojson_files/bayarea.geojson")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant