# Join

Source: https://www.tensorplay.cn/docs/generated/tensorplay.distributed.algorithms.Join.html

```python
class tensorplay.distributed.algorithms.Join(joinables: list[Joinable], enable: bool = True, throw_on_early_termination: bool = False, **kwargs)
```

This class defines the generic join context manager, which allows custom hooks to be called after a process joins.

These hooks should shadow the
collective communications of non-joined processes to prevent hanging and
erroring and to ensure algorithmic correctness. Refer to [JoinHook](/docs/generated/tensorplay.distributed.algorithms.JoinHook.html#tensorplay.distributed.algorithms.JoinHook)
for details about the hook definition.

> **Warning**
>
> The context manager requires each participating Joinable to call the method notify_join_context() before its own per- iteration collective communications to ensure correctness.

Parameters:

- joinables (List[[Joinable](/docs/generated/tensorplay.distributed.algorithms.Joinable.html#tensorplay.distributed.algorithms.Joinable)]) – a list of the participating [Joinable](/docs/generated/tensorplay.distributed.algorithms.Joinable.html#tensorplay.distributed.algorithms.Joinable) s; their hooks are iterated over in the given order.

- enable ([bool](https://docs.python.org/3/builtins/functions.html#bool)) – a flag enabling uneven input detection; setting to False disables the context manager’s functionality and should only be set when the user knows the inputs will not be uneven (default: True).

- throw_on_early_termination ([bool](https://docs.python.org/3/builtins/functions.html#bool)) – a flag controlling whether to throw an exception upon detecting uneven inputs (default: False).

```python
static notify_join_context(joinable: Joinable)
```

Notifies the join context manager that the calling process has not yet joined.

Then, if throw_on_early_termination=True, checks if uneven inputs have been detected
(i.e. if one process has already joined) and throws an exception if so.

This method should be called from a [Joinable](/docs/generated/tensorplay.distributed.algorithms.Joinable.html#tensorplay.distributed.algorithms.Joinable) object before
its per-iteration collective communications. For example, this should
be called at the beginning of the forward pass in
DistributedDataParallel.

Only the first [Joinable](/docs/generated/tensorplay.distributed.algorithms.Joinable.html#tensorplay.distributed.algorithms.Joinable) object passed into the context
manager performs the collective communications in this method, and
for the others, this method is vacuous.

Parameters:

joinable ([Joinable](/docs/generated/tensorplay.distributed.algorithms.Joinable.html#tensorplay.distributed.algorithms.Joinable)) – the [Joinable](/docs/generated/tensorplay.distributed.algorithms.Joinable.html#tensorplay.distributed.algorithms.Joinable) object calling this
method.

Returns:

An async work handle for the all-reduce meant to notify the context
manager that the process has not yet joined if joinable is the
first one passed into the context manager; None otherwise.