gloo_init_parallel_env

paddle.distributed. gloo_init_parallel_env ( rank_id, rank_num, server_endpoint ) [源代码]

该函数仅初始化 GLOO 上下文用于 CPU 间的通信。

参数

  • rank_id (int) - 当前 rank 的编号；

  • rank_num (int) - 当前并行环境中 rank 的总数；

  • server_endpoint (str) - 用于初始化 gloo 上下文的服务器端点，格式为 ip:port。

返回

无

代码示例

>>> import paddle
>>> import multiprocessing
>>> from contextlib import closing
>>> import socket

>>> port_set = set()

>>> def find_free_port():
...     def _free_port():
...         with closing(socket.socket(socket.AF_INET,
...             socket.SOCK_STREAM)) as s:
...             s.bind(('', 0))
...             return s.getsockname()[1]
...     while True:
...         port = _free_port()
...         if port not in port_set:
...             port_set.add(port)
...             return port

>>> def test_gloo_init(id, rank_num, server_endpoint):
...     paddle.distributed.gloo_init_parallel_env(
...         id, rank_num, server_endpoint)

>>> def test_gloo_init_with_multiprocess(num_of_ranks):
...     jobs = []
...     server_endpoint = "127.0.0.1:%s" % (find_free_port())
...     for id in range(num_of_ranks):
...         p = multiprocessing.Process(
...             target=test_gloo_init,
...             args=(id, num_of_ranks, server_endpoint))
...         jobs.append(p)
...         p.start()
...     for proc in jobs:
...         proc.join()

>>> if __name__ == '__main__':
...     # Arg: number of ranks (processes)
...     test_gloo_init_with_multiprocess(2)