PackageImporter

class tensorplay.package.PackageImporter(file_or_buffer: str | ~os.PathLike[str] | ~typing.IO[bytes] | ~tensorplay.package._archive.PackageFileReader, module_allowed: ~collections.abc.Callable[[str], bool] = <function PackageImporter.<lambda>>)[source]

Importers allow you to load code written to packages by PackageExporter. Code is loaded in a hermetic way, using files from the package rather than the normal python import system. This allows for the packaging of model code and data so that it can be run on a server or used in the future for transfer learning.

The importer for packages ensures that code in the module can only be loaded from within the package, except for modules explicitly listed as external during export. The file extern_modules in the zip archive lists all the modules that a package externally depends on. This prevents “implicit” dependencies where the package runs locally because it is importing a locally-installed package, but then fails when the package is copied to another machine.

file_structure(*, include: GlobPattern = '**', exclude: GlobPattern = ()) → Directory[source]

Returns a file structure representation of package’s zipfile.

Parameters:

• include (list[str] | str) – An optional string e.g. "my_package.my_subpackage", or optional list of strings for the names of the files to be included in the zipfile representation. This can also be a glob-style pattern, as described in PackageExporter.mock()

• exclude (list[str] | str) – An optional pattern that excludes files whose name match the pattern.

Returns:

Directory

get_name(obj: Any, name: str | None = None) → tuple[str, str]

Given an object, return a name that can be used to retrieve the object from this environment.

Parameters:

• obj – An object to get the module-environment-relative name for.

• name – If set, use this name instead of looking up __name__ or __qualname__ on obj. This is only here to match how Pickler handles __reduce__ functions that return a string, don’t use otherwise.

Returns:

A tuple (parent_module_name, attr_name) that can be used to retrieve obj from this environment. To use it:

mod = importer.import_module(parent_module_name)
obj = getattr(mod, attr_name)

Raises:

• ObjNotFoundError – we couldn’t retrieve obj by name.

• ObjMisMatchError – we found a different object with the same name as obj.

id()[source]

Returns internal identifier that tensorplay.package uses to distinguish PackageImporter instances. Looks like:

<tensorplay_package_0>

import_module(name: str, package=None)[source]

Load a module from the package if it hasn’t already been loaded, and then return the module. Modules are loaded locally to the importer and will appear in self.modules rather than sys.modules.

Parameters:

• name (str) – Fully qualified name of the module to load.

• package ([type], optional) – Unused, but present to match the signature of importlib.import_module. Defaults to None.

Returns:

The (possibly already) loaded module.

Return type:

types.ModuleType

load_binary(package: str, resource: str) → bytes[source]

Load raw bytes.

Parameters:

• package (str) – The name of module package (e.g. "my_package.my_subpackage").

• resource (str) – The unique name for the resource.

Returns:

The loaded data.

Return type:

bytes

load_pickle(package: str, resource: str, map_location=None) → Any[source]

Unpickles the resource from the package, loading any modules that are needed to construct the objects using import_module().

Parameters:

• package (str) – The name of module package (e.g. "my_package.my_subpackage").

• resource (str) – The unique name for the resource.

• map_location – Retained for interface compatibility; tensorplay embeds tensor payloads directly in the pickle stream, so no remapping of storage records takes place. Defaults to None.

Returns:

The unpickled object.

Return type:

Any

load_text(package: str, resource: str, encoding: str = 'utf-8', errors: str = 'strict') → str[source]

Load a string.

Parameters:

• package (str) – The name of module package (e.g. "my_package.my_subpackage").

• resource (str) – The unique name for the resource.

• encoding (str, optional) – Passed to decode. Defaults to 'utf-8'.

• errors (str, optional) – Passed to decode. Defaults to 'strict'.

Returns:

The loaded text.

Return type:

str

python_version()[source]

Returns the version of python that was used to create this package.

Note: this function is experimental and not Forward Compatible. The plan is to move this into a lock file later on.

Returns:

str | None a python version e.g. 3.8.9 or None if no version was stored with this package

whichmodule(obj: Any, name: str) → str

Find the module name an object belongs to.

This should be considered internal for end-users, but developers of an importer can override it to customize the behavior.

Based on the classic pickle.py approach, but modified to exclude the search into sys.modules.