Conversation
|
squash的pykit不放pr里吧...直接推到master应该没问题 |
|
代码格式整理带来的差异有点多...等你弄完了我整理下再看. |
| GB = 1024**3 | ||
|
|
||
| config = TransferConfig(multipart_threshold=1 * GB, | ||
| multipart_chunksize=512 * MB, ) |
There was a problem hiding this comment.
multipart_chunksize 由原来的256MB 改成了 512MB了
原因：之前发生过备份备份文件失败的情况，经过排查，发现是由于备份文件大，导致分配数量超过了限制。所以先将分片大小调大一些
| @@ -20,21 +20,17 @@ | |||
| import sys | |||
| import time | |||
There was a problem hiding this comment.
这个文件，基本上就是把boto相关的操作替换成了botoclient类。然后用pyauto整理了一下格式
There was a problem hiding this comment.
恩...我看出来了...就是...现在代码不改了吧? 如果不改了我重新整理下, 然后需要强推下你这个分支.
There was a problem hiding this comment.
我来吧，我先把pykit 推到master 上，已经推完了，然后把格式整理和我改的代码分成两个提交点
breakwang
force-pushed
the
bk.refine-mysqlbackup
branch
from
264b1e0 to
d4b51ad
Compare
mysqlops.py
Outdated
| for port in ports: | ||
| jm.put(port) | ||
|
|
||
| jm.join() |
|
|
||
| for port in ports: | ||
| jm.put(port) | ||
| check_backup_file(ports) |
There was a problem hiding this comment.
if和else里都有check_backup_file?
There was a problem hiding this comment.
if 的时候是单独只检查备份文件， else 的时候是 执行恢复的时候也默认检查一下
There was a problem hiding this comment.
如果都检查的话, 应该放在if之前会比较合理...
mysqlops.py
Outdated
| jm = jobq.JobManager([(worker, args.jobs), | ||
| (output, 1)]) | ||
| if cmd == 'check_backup_file': | ||
| check_backup_file(ports) |
There was a problem hiding this comment.
cmd的处理为啥不跟上面的一组ifel...放一起?
There was a problem hiding this comment.
这个函数的参数是所有端口ports，不需要放到jobq里去处理
There was a problem hiding this comment.
所有ports跟正常流程走一样的逻辑比较好, 允许筛选.
命令行上不指定ports就是全部的. 偶尔可能还有需要只检查几个ports的需要
There was a problem hiding this comment.
嗯嗯，可能我表述有误，这个函数的参数是所有端口ports，不需要放到jobq里去处理
这里的所有端口是指--port 参数给定的端口
我这里的逻辑是，恢复--port参数给定的端口之前，先全检查一遍，或者支持使用--check_backup_file 只检查指定的端口操作， 这么可以避免某个端口失败导致整个恢复流程失败。
| setdef(conf, 'clean_after_restore', args.clean_after_restore) | ||
|
|
||
| conf.setdefault('date_str', date_str) | ||
| setdef(conf, 'date_str', args.date_str) |
There was a problem hiding this comment.
为啥有啥用setdef, 有时用dict.setdefault?
There was a problem hiding this comment.
这块逻辑是从原来的主函数里拎出的
dict.setdefault 是为了给date_str 默认赋值
setdef 是更新操作
def setdef(dic, key, v):
if v is not None:
dic[key] = v
d4b51ad to
9335afc
Compare
breakwang
force-pushed
the
bk.refine-mysqlbackup
branch
2 times, most recently
from
17de34c to
18c94a3
Compare
breakwang
requested review from
drmingdrmer
and removed request for
drmingdrmer
breakwang
force-pushed
the
bk.refine-mysqlbackup
branch
from
18c94a3 to
580cc95
Compare
2 participants
see #6