MimicTalk v1.1中文版离线整合包

崎山小鹿

训练lishimeimei_48s_clear

训练1万次
python inference/train_mimictalk_on_a_video.py --video_id data/raw/examples/lishimeimei_48s_clear.mp4 --max_updates 10000 --work_dir checkpoints_mimictalk/lishimeimei_48s_clear


测试生成视频
python inference/mimictalk_infer.py --drv_aud data/raw/examples/jinshuangshi_nv_15s.mp3 --drv_pose data/raw/examples/lishimeimei_48s_clear.mp4 --drv_style data/raw/examples/lishimeimei_48s_clear.mp4 --bg_img data/raw/examples/bg.png --torso_ckpt checkpoints_mimictalk/lishimeimei_48s_clear  --out_name infer_out/tmp/lishimeimei_48s_clear_jinshuangshi.mp4 --out_mode final


测试结果：
嘴型效果不错，清晰度也不错，头发的摆动有阴影，牙齿有拉丝现象，

建议继续训练2000
python inference/train_mimictalk_on_a_video.py --video_id data/raw/examples/lishimeimei_48s_clear.mp4    --torso_ckpt checkpoints_mimictalk/lishimeimei_48s_clear    --max_updates 2000 --work_dir checkpoints_mimictalk/lishimeimei_48s12k_clear

测试生成视频：
python inference/mimictalk_infer.py --drv_aud data/raw/examples/jinshuangshi_nv_15s.mp3 --drv_pose data/raw/examples/liushiqi_15s_clear.mp4 --drv_style data/raw/examples/lishimeimei_48s_clear.mp4 --bg_img data/raw/examples/bg.png --torso_ckpt checkpoints_mimictalk/lishimeimei_48s12k_clear  --out_name infer_out/tmp/lishimeimei_48s12k_clear_jinshuangshi.mp4 --out_mode final

测试结果：清晰度进一步提高，用新的post视频文件解决了头部的摇动幅度比较大的问题。牙齿拉丝明显改善，但是牙齿还不够清晰细腻。训练暂时终止。

崎山小鹿

cp checkpoints/mimictalk_orig/os_secc2plane_torso/config.yaml checkpoints_mimictalk/xiaoqiao_171s_clear
Traceback (most recent call last):
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\inference\app_mimictalkcn.py", line 66, in train_once_args
    trainer = LoRATrainer(inp)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\inference\train_mimictalk_on_a_video.py", line 86, in __init__
    with open(os.path.join(self.inp['work_dir'], 'config.yaml'), "a") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints_mimictalk/xiaoqiao_171s_clear\\config.yaml'
Training ERROR: [Errno 2] No such file or directory: 'checkpoints_mimictalk/xiaoqiao_171s_clear\\config.yaml'

崎山小鹿

切换模型：
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 403, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\fastapi\applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\middleware\errors.py", line 187, in __call__
    raise exc
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\middleware\errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\gradio\route_utils.py", line 766, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\gradio\route_utils.py", line 782, in simple_response
    await self.app(scope, receive, send)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\middleware\exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\starlette\routing.py", line 73, in app
    response = await f(request)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\fastapi\routing.py", line 297, in app
    raw_response = await run_endpoint_function(
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\fastapi\routing.py", line 210, in run_endpoint_function
    return await dependant.call(**values)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\gradio\routes.py", line 1092, in component_server
    return fn(body.data)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\gradio\components\file_explorer.py", line 181, in ls
    full_subdir_path = self._safe_join(subdirectory)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\gradio\components\file_explorer.py", line 212, in _safe_join
    x = safe_join(self.root_dir, combined_path)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\gradio\utils.py", line 1486, in safe_join
    raise InvalidPathError()
gradio.exceptions.InvalidPathError

<gradio.components.file_explorer.FileExplorer object at 0x000002727026E790>

崎山小鹿

ERROR:asyncio:Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
Traceback (most recent call last):
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\asyncio\events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\asyncio\proactor_events.py", line 162, in _call_connection_lost
    self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] 远程主机强迫关闭了一个现有的连接。

崎山小鹿

['checkpoints_mimictalk\\German_20s10k\\model_ckpt_steps_10000.ckpt', 'checkpoints_mimictalk\\lishimeimei_48s12k_clear\\model_ckpt_steps_2000.ckpt', 'checkpoints_mimictalk\\liushiqi_130s12k_clear\\model_ckpt_steps_2000.ckpt', 'checkpoints_mimictalk\\liushiqi_130s20k_clear\\model_ckpt_steps_2000.ckpt', 'checkpoints_mimictalk\\liushiqi_130s20k_clear\\model_ckpt_steps_4000.ckpt', 'checkpoints_mimictalk\\liushiqi_130s20k_clear\\model_ckpt_steps_6000.ckpt', 'checkpoints_mimictalk\\liushiqi_130s20k_clear\\model_ckpt_steps_8000.ckpt', 'checkpoints_mimictalk\\xiaoqiao_30s2k\\model_ckpt_steps_2000.ckpt', 'checkpoints_mimictalk\\yumo_76s12k_clear\\model_ckpt_steps_10000.ckpt', 'checkpoints_mimictalk\\yumo_76s12k_clear\\model_ckpt_steps_2000.ckpt', 'checkpoints_mimictalk\\yumo_76s12k_clear\\model_ckpt_steps_4000.ckpt', 'checkpoints_mimictalk\\yumo_76s12k_clear\\model_ckpt_steps_6000.ckpt', 'checkpoints_mimictalk\\yumo_76s12k_clear\\model_ckpt_steps_8000.ckpt']

崎山小鹿

File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\gradio\components\file_explorer.py", line 214, in _safe_join
    x = safe_join(self.root_dir, combined_path)
  File "G:\ProgramData\anaconda3\envs\MimicTalk39\lib\site-packages\gradio\utils.py", line 1486, in safe_join
    raise InvalidPathError()
gradio.exceptions.InvalidPathError

Root Directory: G:\ProgramData\anaconda3\envs\MimicTalk39
Combined Path: checkpoints_mimictalk\liushiqi_130s12k_clear

崎山小鹿

训练模型的时候提示错误：
File "G:\MimicTalkV1.1\inference\train_mimictalk_on_a_video.py", line 218, in load_training_data
    ids = convert_to_tensor(coeff_dict['id']).reshape([-1, 80]).cuda()  # 身份特征
TypeError: 'bool' object is not subscriptable