Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tools/train_net.py FAILED #390

Open
taokunade-gt opened this issue Mar 26, 2023 · 1 comment
Open

tools/train_net.py FAILED #390

taokunade-gt opened this issue Mar 26, 2023 · 1 comment

Comments

@taokunade-gt
Copy link

(fcos) C:\Users\Okunade\Desktop\FCOS-MASTER\FCOS>python -m torch.distributed.launch \ --nproc_per_node=2 \ tools/train_net.py \ --config-file configs/fcos/fcos_imprv_R_50_FPN_1x.yaml \ DATALOADER.NUM_WORKERS 2 \ OUTPUT_DIR training_dir/fcos_imprv_R_50_FPN_1x
NOTE: Redirects are currently not supported in Windows or MacOs.
C:\Users\Okunade\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\distributed\launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects --local_rank argument to be set, please
change it to read from os.environ['LOCAL_RANK'] instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions

warnings.warn(
C:\Users\Okunade\AppData\Local\Programs\Python\Python39\python.exe: can't find 'main' module in 'C:\'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 13636) of binary: C:\Users\Okunade\AppData\Local\Programs\Python\Python39\python.exe
Traceback (most recent call last):
File "C:\Users\Okunade\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Okunade\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\Okunade\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\distributed\launch.py", line 193, in
main()
File "C:\Users\Okunade\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\distributed\launch.py", line 189, in main
launch(args)
File "C:\Users\Okunade\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\distributed\launch.py", line 174, in launch
run(args)
File "C:\Users\Okunade\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\distributed\run.py", line 710, in run
elastic_launch(
File "C:\Users\Okunade\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\distributed\launcher\api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "C:\Users\Okunade\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\distributed\launcher\api.py", line 258, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

tools/train_net.py FAILED

Failures:
<NO_OTHER_FAILURES>

Root Cause (first observed failure):
[0]:
time : 2023-03-26_12:13:34
host : LAPTOP-DGQ846FJ
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 13636)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

@kvnptl
Copy link

kvnptl commented Sep 3, 2023

Getting the same, did you solve this one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants