API Documentation

Read / Write functions

uriutils.uriutils.uri_open(uri, mode='rb', auto_compress=True, in_memory=True, delete_tempfile=True, textio_args={}, storage_args={})

Opens a URI for reading / writing. Analogous to the open() function. This method supports with context handling:

with uri_open('http://www.example.com', mode='r') as f:
    print(f.read())

Parameters:

• uri (str) – URI of file to open
• mode (str) – Either rb, r, w, or wb for read/write modes in binary/text respectiely
• auto_compress (bool) – Whether to automatically use the gzip module with .gz URIsF
• in_memory (bool) – Whether to store entire file in memory or in a local temporary file
• delete_tempfile (bool) – When in_memory is False, whether to delete the temporary file on close
• textio_args (dict) – Keyword arguments to pass to io.TextIOWrapper for text read/write mode
• storage_args (dict) – Keyword arguments to pass to the underlying storage object

Returns:

file-like object to URI

uriutils.uriutils.uri_read(*args, **kwargs)

Reads the contents of a URI into a string or bytestring. See uri_open() for complete description of keyword parameters.

Returns:Contents of URI Return type:str, bytes

uriutils.uriutils.uri_dump(uri, content, mode='wb', **kwargs)

Dumps the contents of a string/bytestring into a URI. See uri_open() for complete description of keyword parameters.

Parameters:

• uri (str) – URI to dump contents to
• content (str) – Contents to write to URI
• mode (str) – Either w, or wb to write binary/text content respectiely

URI information

uriutils.uriutils.uri_exists(uri, storage_args={})

Check if URI exists.

Parameters:

• uri (str) – URI to check existence
• storage_args (dict) – Keyword arguments to pass to the underlying storage object

Returns:

True if URI exists

Return type:

bool

uriutils.uriutils.uri_exists_wait(uri, timeout=300, interval=5, storage_args={})

Block / waits until URI exists.

Parameters:

• uri (str) – URI to check existence
• timeout (float) – Number of seconds before timing out
• interval (float) – Calls uri_exists() every interval seconds
• storage_args (dict) – Keyword arguments to pass to the underlying storage object

Returns:

True if URI exists

Return type:

bool

uriutils.uriutils.get_uri_metadata(uri, storage_args={})

Get the “metadata” from URI. This is most commonly used with bucket storage on the Cloud such as S3 and Google Cloud.

Parameters:

• uri (str) – URI to get metadata for
• storage_args (dict) – Keyword arguments to pass to the underlying storage object

Returns:

Metadata associated with URI

Return type:

dict

uriutils.uriutils.get_uri_obj(uri, storage_args={})

Retrieve the underlying storage object based on the URI (i.e., scheme).

Parameters:

• uri (str) – URI to get storage object for
• storage_args (dict) – Keyword arguments to pass to the underlying storage object

Argument Parser types

class uriutils.uriutils.URIType

A convenience class that can be used as the type argument to argparse.ArgumentParser.add_argument(). It will return the result of urllib.parse.urlparse().

class uriutils.uriutils.URIFileType(mode='rb', **kwargs)

A convenience class that can be used as the type argument to argparse.ArgumentParser.add_argument(). It will return a file-like object using uri_open().

See uri_open() for complete description of keyword parameters.

class uriutils.uriutils.URIDirType(create=False, storage_args={})

A convenience class that can be used as the type argument to argparse.ArgumentParser.add_argument(). It will return the result of urllib.parse.urlparse().

Parameters:

• create (bool) – Whether to create directory (and thus “ensure” that directory exists)
• storage_args (dict) – Keyword arguments to pass to the underlying storage object