Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[自荐][工具] 一款可以本地实时生成字幕的视频播放器 YPlayer #1593

Open
yujinqiu opened this issue Jan 14, 2021 · 35 comments

Comments

@yujinqiu
Copy link

yujinqiu commented Jan 14, 2021

背景

在学习外语技术相关视频的时候特别喜欢 youtube 上的字幕, 但是经常遇到一些问题

  1. 并不是所有的 youtube 视频都有字幕
  2. 有时候视频就在本地, 如果要上传到 youtube 上的话, 网络是一个瓶颈

当然业界也有不少人在开发调用公有云的语音识别接口的播放器,
考虑到公有云语音识别的费用和数据的隐私性的问题,
我开发了一个本地实时生成字幕的视频播放器 YPlayer,
也就是说整个字幕生成过程不需要任何网络请求, 大家可以关闭 Wifi 体验一下.

下载地址

YPlayer Appstore
充分尊重大家的数据隐私性, 不采集任何数据.
image

特殊说明

由于该播放器使用的是 SwiftUI 的比较新的特性, 所以目前只能在 macOS Big Sur (>= 11.0) 版本上运行.

@ScienceZone
Copy link

ScienceZone commented Jan 14, 2021 via email

@liudhzhyym
Copy link

亲测好用,啥时候有ios版本?

@yujinqiu
Copy link
Author

@liudhzhyym iOS 版本在审核中, 应该快了.

@yujinqiu
Copy link
Author

@ScienceZone 暂时没有人力支持 windows, 抱歉.

@HammerChen
Copy link

还没敢更新Big Sur😭😭

@yili1992
Copy link

为此决定升级系统试试

@Lin-huai
Copy link

不能下载啊

@lieliedurgan
Copy link

背景

在学习外语技术相关视频的时候特别喜欢 youtube 上的字幕, 但是经常遇到一些问题

  1. 并不是所有的 youtube 视频都有字幕
  2. 有时候视频就在本地, 如果要上传到 youtube 上的话, 网络是一个瓶颈

当然业界也有不少人在开发调用公有云的语音识别接口的播放器,
考虑到公有云语音识别的费用和数据的隐私性的问题,
我开发了一个本地实时生成字幕的视频播放器 YPlayer,
也就是说整个字幕生成过程不需要任何网络请求, 大家可以关闭 Wifi 体验一下.

下载地址

YPlayer Appstore
充分尊重大家的数据隐私性, 不采集任何数据.
image

特殊说明

由于该播放器使用的是 SwiftUI 的比较新的特性, 所以目前只能在 macOS Big Sur (>= 11.0) 版本上运行.

国内的商店没有上架吗?

@Lin-huai
Copy link

Lin-huai commented Jan 22, 2021 via email

@yujinqiu
Copy link
Author

@Lin-huai 国内 Appstore 可以下载. 麻烦确认一下系统是否是 Big Sur ?

@yujinqiu
Copy link
Author

为此决定升级系统试试

@yili1992 感谢支持

@Lin-huai
Copy link

Lin-huai commented Jan 22, 2021 via email

@hallkk
Copy link

hallkk commented Jan 22, 2021

求个其他平台的版本。。

@tunshiyu
Copy link

为此决定升级系统试试

如果你是开发,不要升级。挺鸡肋。

@Lin-huai
Copy link

Lin-huai commented Jan 23, 2021 via email

@yujinqiu
Copy link
Author

@tunshiyu @Lin-huai 我一直相信作为开发者肯定是个讲逻辑和理性的群体, 升不升级 Big Sur 大家自己选择判断, 个人不做评价, 毕竟 Big Sur 的市场占有率不是我的 KPI :-)

开发 YPlayer 的初心是自己作为一个开发者在学习新技术过程中遇到的问题, 但是没有找到一个比较好的解决方案, 所以自己开发了一个, 希望能帮助遇到类似问题的其他人, 仅此而已.

至于 Big Sur 的问题导致 YPlayer 无法使用的话, 那只能说明字幕这个需求对你来说不是个很强的痛点, 那自己就不要升级就好了, 没有什么关系的.

@sbilly
Copy link

sbilly commented Jan 25, 2021

加上自动翻译吧。

@bigyelow
Copy link

是用的 SFSpeechRecognizer 吗?

@lian-tian
Copy link

lian-tian commented Jan 25, 2021 via email

@bigyelow
Copy link

我用iOS。商商还没有。👍

@lian-tian 什么意思?

@yujinqiu
Copy link
Author

@sbilly 内部有一个 iOS 翻译的版本, 目前的效果不太好, 有些核心问题还没有解决, 还没有打算放出来.

@lian-tian
Copy link

lian-tian commented Jan 26, 2021 via email

@yili1992
Copy link

yili1992 commented Jan 29, 2021

为此决定升级系统试试

@yili1992 感谢支持

看起来一开始要下载会儿语言模型

@yujinqiu
Copy link
Author

yujinqiu commented Feb 1, 2021

@yili1992 是的, 因为在本地翻译, 肯定要下载语言模型的, 不然就是无米之炊了.

@yili1992
Copy link

yili1992 commented Feb 1, 2021

@yili1992 是的, 因为在本地翻译, 肯定要下载语言模型的, 不然就是无米之炊了.

现在看起来是自动生成字幕,有打算增加下 自动翻译的功能吗?

@yujinqiu
Copy link
Author

yujinqiu commented Feb 2, 2021

@yili1992 内部其实已经有一个自动翻译功能, 只能看一个大概, 效果不太满意, 目前还没有打算放出来. 因为语音识别的准确性和多语言断句上严重影响翻译的效果, 还在研究中.
断句问题是因为语音识别出来并没有带上标点符号, 虽然可以通过语音的停顿来简单分句, 但是很多时候语速快的情况下, 效果并不好.
断句效果不好, 翻译效果就更不好.
大家有相关的思路, 方法也可以推荐给我.

@yujinqiu
Copy link
Author

yujinqiu commented Feb 4, 2021

@lian-tian @liudhzhyym @yili1992 @bigyelow iOS 版本在一波三折之后终于上架, 目前还是免费状态, 后续 iOS 会考虑收费.

https://apps.apple.com/cn/app/yplayer/id1542569291?l=en

@AllocAndInit
Copy link

@yujinqiu 你好,请问一下,这个内部的大致实现原理是怎么样的

@claviering
Copy link

Using Live Caption in Google Chrome

Setting -> Advanced -> Accessibility -> Turn on Live Caption
or search Live Caption

image

@yujinqiu
Copy link
Author

yujinqiu commented Aug 2, 2022

Using Live Caption in Google Chrome

Setting -> Advanced -> Accessibility -> Turn on Live Caption or search Live Caption

image

Google's live captioning in Chrome is amazing, if your use case is just inside Chrome and only have English live caption, use it, it pretty good actually.

But if want to have live caption more than Chrome and have other language support (French, German, Mandarin, etc) and other app (Zoom, Teams, IINA ... ) you can give BeMyEars a try :-)

@baixiaoji
Copy link

@yujinqiu M1 OS: 13.5.2 ventura打开 app crash

@yujinqiu
Copy link
Author

yujinqiu commented Nov 2, 2023

@baixiaoji 方便给一下 崩溃报告 吗? 在系统的 控制台 里面就可以看到.

@baixiaoji
Copy link

@baixiaoji 方便给一下 崩溃报告 吗? 在系统的 控制台 里面就可以看到.
这些可以吗?
Termination Reason: Namespace DYLD, Code 4 Symbol missing
Symbol not found: _$s9VisionKit32ImageAnalysisOverlayViewDelegatePAAE07overlayF0_11needsUpdateyAA0cdeF0C_So6NSMenuCtF
Referenced from: /Applications/YPlayer.app/Contents/MacOS/YPlayer
Expected in: /System/Library/Frameworks/VisionKit.framework/Versions/A/VisionKit
(terminated at launch; ignore backtrace)

Thread 0 Crashed:
0 dyld 0x19ce9d118 __abort_with_payload + 8
1 dyld 0x19cea8d7c abort_with_payload_wrapper_internal + 104
2 dyld 0x19cea8db0 abort_with_payload + 16
3 dyld 0x19ce348a8 dyld4::halt(char const*) + 328
4 dyld 0x19ce31590 dyld4::prepare(dyld4::APIs&, dyld3::MachOAnalyzer const*) + 4184
5 dyld 0x19ce2fe18 start + 1964

Thread 0 crashed with ARM Thread State (64-bit):
x0: 0x0000000000000006 x1: 0x0000000000000004 x2: 0x000000016fcb4390 x3: 0x00000000000000ed
x4: 0x000000016fcb3f90 x5: 0x0000000000000000 x6: 0x0000000000000000 x7: 0x000000016fcb3978
x8: 0x0000000000000020 x9: 0x0000000000000009 x10: 0x0000000000000001 x11: 0x000000000000000a
x12: 0x0000000000000000 x13: 0x0000000000000039 x14: 0x00000002488335c9 x15: 0x0000000000008000
x16: 0x0000000000000209 x17: 0x000000019ce2d35c x18: 0x0000000000000000 x19: 0x0000000000000000
x20: 0x000000016fcb3f90 x21: 0x00000000000000ed x22: 0x000000016fcb4390 x23: 0x0000000000000004
x24: 0x0000000000000006 x25: 0x00000000000012c0 x26: 0x0000000000000001 x27: 0x000000001f070004
x28: 0x00000001012e0000 fp: 0x000000016fcb3f60 lr: 0x000000019cea8d7c
sp: 0x000000016fcb3f20 pc: 0x000000019ce9d118 cpsr: 0x00001000
far: 0x0000000101320000 esr: 0x56000080 Address size fault

Binary Images:
0x100148000 - 0x101117fff net.duoziwei.yplayer (5.2) /Applications/YPlayer.app/Contents/MacOS/YPlayer
0x10134c000 - 0x10134ffff FFmpeg-Kit (5.2) <4130e9b7-19c2-3994-bcdf-9019224520be> /Applications/YPlayer.app/Contents/Frameworks/FFmpeg-Kit.framework/Versions/A/FFmpeg-Kit
0x101338000 - 0x10133bfff FFmpeg (5.2) /Applications/YPlayer.app/Contents/Frameworks/FFmpeg.framework/Versions/A/FFmpeg
0x101484000 - 0x1014cffff com.arthenica.ffmpegkit.FFmpegKit (5.2) /Applications/YPlayer.app/Contents/Frameworks/ffmpegkit.framework/Versions/A/ffmpegkit
0x1013a4000 - 0x1013affff com.arthenica.ffmpegkit.Libavdevice (5.2) <0bc4bf9a-b2e6-3c84-ae43-f5ac7eb64c94> /Applications/YPlayer.app/Contents/Frameworks/libavdevice.framework/Versions/A/libavdevice
0x101600000 - 0x10164ffff com.arthenica.ffmpegkit.Libavutil (5.2) /Applications/YPlayer.app/Contents/Frameworks/libavutil.framework/Versions/A/libavutil
0x1013c4000 - 0x1013d3fff com.arthenica.ffmpegkit.Libswresample (5.2) /Applications/YPlayer.app/Contents/Frameworks/libswresample.framework/Versions/A/libswresample
0x101cc0000 - 0x101e9ffff com.arthenica.ffmpegkit.Libavfilter (5.2) <5b0ac9ac-4167-3a2e-b4c0-68e531eeed84> /Applications/YPlayer.app/Contents/Frameworks/libavfilter.framework/Versions/A/libavfilter
0x1013e4000 - 0x10142ffff com.arthenica.ffmpegkit.Libswscale (5.2) <97451c17-1198-3060-bd54-3288cb2457ab> /Applications/YPlayer.app/Contents/Frameworks/libswscale.framework/Versions/A/libswscale
0x102f5c000 - 0x10366bfff com.arthenica.ffmpegkit.Libavcodec (5.2) /Applications/YPlayer.app/Contents/Frameworks/libavcodec.framework/Versions/A/libavcodec
0x10177c000 - 0x101a5ffff com.arthenica.ffmpegkit.Libavformat (5.2) /Applications/YPlayer.app/Contents/Frameworks/libavformat.framework/Versions/A/libavformat
0x19ce2a000 - 0x19ceb8587 dyld (*) /usr/lib/dyld

External Modification Summary:
Calls made by other processes targeting this process:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
Calls made by this process:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
Calls made by all processes on this machine:
task_for_pid: 0
thread_create: 0
thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=1.4G resident=0K(0%) swapped_out_or_unallocated=1.4G(100%)
Writable regions: Total=15.4M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=15.4M(100%)

@yujinqiu
Copy link
Author

yujinqiu commented Nov 6, 2023

@baixiaoji 麻烦帮忙试试 https://models.c0nv2sation.uk/YPlayer.zip 这个版本是否修复了.

@baixiaoji
Copy link

@baixiaoji 麻烦帮忙试试 https://models.c0nv2sation.uk/YPlayer.zip 这个版本是否修复了.

还是不行,日志信息

Termination Reason: Namespace DYLD, Code 4 Symbol missing
Symbol not found: _$s9VisionKit32ImageAnalysisOverlayViewDelegatePAAE07overlayF0_11needsUpdateyAA0cdeF0C_So6NSMenuCtF
Referenced from: <3E88CF2B-ABAF-3768-9598-239D858F17B4> /Users/USER/Downloads/YPlayer.app/Contents/MacOS/YPlayer
Expected in: /System/Library/Frameworks/VisionKit.framework/Versions/A/VisionKit
(terminated at launch; ignore backtrace)

Thread 0 Crashed:
0 dyld 0x1a1af5118 __abort_with_payload + 8
1 dyld 0x1a1b00d7c abort_with_payload_wrapper_internal + 104
2 dyld 0x1a1b00db0 abort_with_payload + 16
3 dyld 0x1a1a8c8a8 dyld4::halt(char const*) + 328
4 dyld 0x1a1a89590 dyld4::prepare(dyld4::APIs&, dyld3::MachOAnalyzer const*) + 4184
5 dyld 0x1a1a87e18 start + 1964

Thread 0 crashed with ARM Thread State (64-bit):
x0: 0x0000000000000006 x1: 0x0000000000000004 x2: 0x000000016f010380 x3: 0x00000000000000f9
x4: 0x000000016f00ff80 x5: 0x0000000000000000 x6: 0x0000000000000000 x7: 0x000000016f00f968
x8: 0x0000000000000020 x9: 0x0000000000000009 x10: 0x0000000000000001 x11: 0x000000000000000a
x12: 0x0000000000000000 x13: 0x0000000000000033 x14: 0x000000024d48b5c9 x15: 0x0000000000008000
x16: 0x0000000000000209 x17: 0x00000001a1a8535c x18: 0x0000000000000000 x19: 0x0000000000000000
x20: 0x000000016f00ff80 x21: 0x00000000000000f9 x22: 0x000000016f010380 x23: 0x0000000000000004
x24: 0x0000000000000006 x25: 0x00000000000012c0 x26: 0x0000000000000001 x27: 0x000000001f070004
x28: 0x0000000101f70000 fp: 0x000000016f00ff50 lr: 0x00000001a1b00d7c
sp: 0x000000016f00ff10 pc: 0x00000001a1af5118 cpsr: 0x00001000
far: 0x0000000101fb0000 esr: 0x56000080 Address size fault

Binary Images:
0x100dec000 - 0x101dbbfff net.duoziwei.yplayer (5.3) <3e88cf2b-abaf-3768-9598-239d858f17b4> /Users/USER/Downloads/YPlayer.app/Contents/MacOS/YPlayer
0x101fd8000 - 0x101fdbfff FFmpeg-Kit (1.0) /Users/USER/Downloads/YPlayer.app/Contents/Frameworks/FFmpeg-Kit.framework/Versions/A/FFmpeg-Kit
0x101fc8000 - 0x101fcbfff FFmpeg (1.0) <536bc005-1b36-32d7-ac03-67f8c3c67fde> /Users/USER/Downloads/YPlayer.app/Contents/Frameworks/FFmpeg.framework/Versions/A/FFmpeg
0x10210c000 - 0x102157fff com.arthenica.ffmpegkit.FFmpegKit (5.1) /Users/USER/Downloads/YPlayer.app/Contents/Frameworks/ffmpegkit.framework/Versions/A/ffmpegkit
0x103198000 - 0x1038a7fff com.arthenica.ffmpegkit.Libavcodec (59.37.100) /Users/USER/Downloads/YPlayer.app/Contents/Frameworks/libavcodec.framework/Versions/A/libavcodec
0x102030000 - 0x10203ffff com.arthenica.ffmpegkit.Libswresample (4.7.100) /Users/USER/Downloads/YPlayer.app/Contents/Frameworks/libswresample.framework/Versions/A/libswresample
0x102050000 - 0x10209bfff com.arthenica.ffmpegkit.Libswscale (6.7.100) <97451c17-1198-3060-bd54-3288cb2457ab> /Users/USER/Downloads/YPlayer.app/Contents/Frameworks/libswscale.framework/Versions/A/libswscale
0x1028a0000 - 0x102b83fff com.arthenica.ffmpegkit.Libavformat (59.27.100) /Users/USER/Downloads/YPlayer.app/Contents/Frameworks/libavformat.framework/Versions/A/libavformat
0x1020b4000 - 0x1020bffff com.arthenica.ffmpegkit.Libavdevice (59.7.100) <0bc4bf9a-b2e6-3c84-ae43-f5ac7eb64c94> /Users/USER/Downloads/YPlayer.app/Contents/Frameworks/libavdevice.framework/Versions/A/libavdevice
0x102288000 - 0x1022d7fff com.arthenica.ffmpegkit.Libavutil (57.28.100) /Users/USER/Downloads/YPlayer.app/Contents/Frameworks/libavutil.framework/Versions/A/libavutil
0x102404000 - 0x1025e3fff com.arthenica.ffmpegkit.Libavfilter (8.44.100) <5b0ac9ac-4167-3a2e-b4c0-68e531eeed84> /Users/USER/Downloads/YPlayer.app/Contents/Frameworks/libavfilter.framework/Versions/A/libavfilter
0x1a1a82000 - 0x1a1b10587 dyld (*) /usr/lib/dyld

Repository owner locked as too heated and limited conversation to collaborators Nov 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests