比較提交

...

2 次程式碼提交

作者 SHA1 備註 日期
MasterofJOKers c2f5732351 mirror-yt: Handle returncode 1
We switch from `subprocess.run` to manually handling a `Popen` object so
we can decide whether a returncode of 1 is fine or now depending on the
output we see. Since we want live output during the download, we cannot
just capture all the output. Instead, we have to regularly print
whatever we captured. Therefore, we need a `Popen` object and need to
call `communicate()` with a timeout on it.
2024-04-21 23:18:54 +02:00
MasterofJOKers 16de972f7b mirror-yt: Add a sleep between downloads
We want to play nice in the internet.
2024-04-21 23:18:19 +02:00
共有 1 個檔案被更改,包括 28 行新增2 行删除

查看文件

@ -1,6 +1,7 @@
#!/usr/bin/env python3
from pathlib import Path
import subprocess
import time
import click
@ -25,7 +26,7 @@ def download(allow_unknown, base_dir, channel_file):
line = line.strip()
if line.startswith('#'):
continue
dir_name, url = line.split(';')
channels[dir_name] = url
@ -59,7 +60,32 @@ def download(allow_unknown, base_dir, channel_file):
'--write-thumbnail',
url
]
subprocess.run(cmd, check=True, cwd=target_dir)
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=target_dir)
prev_output = ''
while p.poll() is None:
try:
stdout_data, stderr_data = p.communicate(timeout=0.5)
except subprocess.TimeoutExpired as e:
if e.output:
print(e.output[len(prev_output):].decode('utf-8'), end='')
prev_output = e.output or ''
else:
print(stdout_data[len(prev_output):].decode('utf-8'))
stdout_lines = stdout_data.decode('utf-8').splitlines()
if p.returncode == 0:
return
if p.returncode == 101:
pass
elif p.returncode == 1 and stdout_lines[-1] and 'Finished downloading playlist' in stdout_lines[-1]:
pass
else:
breakpoint()
raise subprocess.CalledProcessError(returncode=p.returncode, cmd=cmd)
sleep_seconds = 5
print(f"Sleeping {sleep_seconds}s")
time.sleep(sleep_seconds)
if __name__ == '__main__':