Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DRM is not being removed: Calibre 4.23/DeDRM6.8/Kindle for Mac 1.26.1/Mac Big Sur #1369

Closed
ghost opened this issue Nov 20, 2020 · 41 comments
Closed

Comments

@ghost
Copy link

ghost commented Nov 20, 2020

CheckList

  • [x ] The Title and The Log Title are setted correctly.
  • [ x] Clarified about my environment.
  • [ x] Code block is used for the log.

Title

DRM is not being removed

My Environment

Calibre: 4.23

Kindle: 1.26.1

DeDRM: 6.8

Mac mini M1 Big Sur
image

Log

Removing DRM of Salt to the Sea
calibre Debug log
calibre 4.23  embedded-python: True is64bit: True
Darwin-20.1.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '20.1.0', 'Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:10 PDT 2020; root:xnu-7195.50.7~2/RELEASE_ARM64_T8101')
Python 2.7.16
OSX: ('10.16', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: DeDRM (6, 8, 0)
calibre 4.23  embedded-python: True is64bit: True
Darwin-20.1.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '20.1.0', 'Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:10 PDT 2020; root:xnu-7195.50.7~2/RELEASE_ARM64_T8101')
Python 2.7.16
OSX: ('10.16', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: DeDRM (6, 8, 0)
devicePixelRatio: 1.0
logicalDpi: 72.0 x 72.0
physicalDpi: 92.0000013817 x 91.0000013667
Using calibre Qt style: True
[0.00] Starting up...
[0.01] Showing splash screen...
[0.21] splash screen shown
[0.21] Initializing db...
[0.22] db initialized
[0.22] Constructing main UI...
[0.80] main UI initialized...
[0.80] Hiding splash screen
[8.18] splash screen hidden
[8.18] Started up in 8.18 seconds with 253 books
DeDRM v6.8.0: Trying to decrypt B00YM34WM8_EBOK.azw
Using Library AlfCrypto Python
MobiDeDrm v0.42.
Copyright © 2008-2017 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: Salt to the Sea
Found 0 keys to try after 0.2 seconds
Crypto Type is: 2
DeDRM v6.8.0: Failed to decrypt with error: No key found in 0 keys tried.
DeDRM v6.8.0: Looking for new default Kindle Key after 0.2 seconds
Found k4Mac kinf2018 file: /Users/lisa/Library/Application Support/Kindle/storage/.kinf2018
Couldn't decrypt file.
DeDRM v6.8.0: Ultimately failed to decrypt after 0.3 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md
Running file type plugin DeDRM failed with traceback:
Traceback (most recent call last):
  File "site-packages/calibre/customize/ui.py", line 172, in _run_filetype_plugins
  File "calibre_plugins.dedrm.__init__", line 625, in run
  File "calibre_plugins.dedrm.__init__", line 575, in KindleMobiDecrypt
DeDRMError: DeDRM v6.8.0: Ultimately failed to decrypt after 0.3 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md
Worker Launch took: 0.0718188285828
Job: 0 Convert book 1 of 1 (Salt to the Sea) finished
Convert book 1 of 1 (Salt to the Sea)
	Conversion options changed from defaults:
	  read_metadata_from_opf: u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_ivwUhU/BhxNOm.opf'
	  verbose: 2
	  cover: u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_ivwUhU/f3MaNO.jpeg'
	  output_profile: u'kindle_oasis'
	  input_profile: u'kindle'
	Resolved conversion options
	calibre version: 4.23.0
	{'asciiize': False,
	 'author_sort': None,
	 'authors': None,
	 'base_font_size': 0.0,
	 'book_producer': None,
	 'change_justification': u'original',
	 'chapter': u"//*[((name()='h1' or name()='h2') and re:test(., '\\s*((chapter|book|section|part)\\s+)|((prolog|prologue|epilogue)(\\s+|$))', 'i')) or @class = 'chapter']",
	 'chapter_mark': u'pagebreak',
	 'comments': None,
	 'cover': u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_ivwUhU/f3MaNO.jpeg',
	 'debug_pipeline': None,
	 'dehyphenate': True,
	 'delete_blank_paragraphs': True,
	 'disable_font_rescaling': False,
	 'dont_split_on_page_breaks': False,
	 'duplicate_links_in_toc': False,
	 'embed_all_fonts': False,
	 'embed_font_family': None,
	 'enable_heuristics': False,
	 'epub_flatten': False,
	 'epub_inline_toc': False,
	 'epub_toc_at_end': False,
	 'epub_version': u'2',
	 'expand_css': False,
	 'extra_css': None,
	 'extract_to': None,
	 'filter_css': u'',
	 'fix_indents': True,
	 'flow_size': 260,
	 'font_size_mapping': None,
	 'format_scene_breaks': True,
	 'html_unwrap_factor': 0.4,
	 'input_encoding': None,
	 'input_profile': <calibre.customize.profiles.KindleInput object at 0x118552090>,
	 'insert_blank_line': False,
	 'insert_blank_line_size': 0.5,
	 'insert_metadata': False,
	 'isbn': None,
	 'italicize_common_cases': True,
	 'keep_ligatures': False,
	 'language': None,
	 'level1_toc': None,
	 'level2_toc': None,
	 'level3_toc': None,
	 'line_height': 0.0,
	 'linearize_tables': False,
	 'margin_bottom': 5.0,
	 'margin_left': 5.0,
	 'margin_right': 5.0,
	 'margin_top': 5.0,
	 'markup_chapter_headings': True,
	 'max_toc_links': 50,
	 'minimum_line_height': 120.0,
	 'no_chapters_in_toc': False,
	 'no_default_epub_cover': False,
	 'no_inline_navbars': False,
	 'no_svg_cover': False,
	 'output_profile': <calibre.customize.profiles.KindleOasisOutput object at 0x118552690>,
	 'page_breaks_before': u"//*[name()='h1' or name()='h2']",
	 'prefer_metadata_cover': False,
	 'preserve_cover_aspect_ratio': False,
	 'pretty_print': True,
	 'pubdate': None,
	 'publisher': None,
	 'rating': None,
	 'read_metadata_from_opf': u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_ivwUhU/BhxNOm.opf',
	 'remove_fake_margins': True,
	 'remove_first_image': False,
	 'remove_paragraph_spacing': False,
	 'remove_paragraph_spacing_indent_size': 1.5,
	 'renumber_headings': True,
	 'replace_scene_breaks': u'',
	 'search_replace': '[]',
	 'series': None,
	 'series_index': None,
	 'smarten_punctuation': False,
	 'sr1_replace': None,
	 'sr1_search': None,
	 'sr2_replace': None,
	 'sr2_search': None,
	 'sr3_replace': None,
	 'sr3_search': None,
	 'start_reading_at': None,
	 'subset_embedded_fonts': False,
	 'tags': None,
	 'timestamp': None,
	 'title': None,
	 'title_sort': None,
	 'toc_filter': None,
	 'toc_threshold': 6,
	 'toc_title': None,
	 'transform_css_rules': '[]',
	 'unsmarten_punctuation': False,
	 'unwrap_lines': True,
	 'use_auto_toc': False,
	 'verbose': 2}
	InputFormatPlugin: MOBI Input running
	on /var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_ivwUhU/FurDhx.azw
	Found KF8 MOBI of type u'standalone'
	Python function terminated unexpectedly: Salt to the Sea
	Traceback (most recent call last):
	  File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 187, in main
	    return run_entry_point()
	  File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 121, in run_entry_point
	    return getattr(pmod, func)()
	  File "site-packages/calibre/utils/ipc/worker.py", line 209, in main
	  File "site-packages/calibre/gui2/convert/gui_conversion.py", line 43, in gui_convert_override
	  File "site-packages/calibre/gui2/convert/gui_conversion.py", line 28, in gui_convert
	  File "site-packages/calibre/ebooks/conversion/plumber.py", line 1110, in run
	  File "site-packages/calibre/customize/conversion.py", line 246, in __call__
	  File "site-packages/calibre/ebooks/conversion/plugins/mobi_input.py", line 47, in convert
	  File "site-packages/calibre/ebooks/mobi/reader/mobi8.py", line 86, in __call__
	  File "site-packages/calibre/ebooks/mobi/reader/mobi6.py", line 158, in check_for_drm
	calibre.ebooks.DRMError: Salt to the Sea
@ElleKayEm
Copy link

Was the book downloaded with the current installation of Kindle for Mac located on the same machine and user as calibre with DeDRM?

@ghost
Copy link
Author

ghost commented Nov 20, 2020

Yes. But not the "current" installation of Kindle for Mac. It was version 1.26.1

@ElleKayEm
Copy link

For some reason, DeDRM is unable to decrypt the key that it is finding. Possibly something left behind from a previous installation. Try deleting this file: /Users/lisa/Library/Application Support/Kindle/storage/.kinf2018
Then remove and re-add the book.

@ghost
Copy link
Author

ghost commented Nov 20, 2020

So I decided to just delete everything and start over. I uninstalled Kindle, Calibre, deDRM plugin. (I also deleted that entire Kindle folder under Application Support (and anywhere else I found Kindle App items).
I reinstalled everything in the versions noted above. Tried to import a book from my Amazon library and received the same error.
I went to the /Users/lisa/Library/Application Support/Kindle/storage/ folder and that .kinf2018 file was there again. So I deleted it (again). When I went back into Kindle for Mac, my "device registration" was deleted, so I had to re-register that.
But it kept crashing. Once I de-registered from Amazon's site, it let me re-register it.
I deleted the book and tried one more time to download and import the book into Calibre, and still received the cannot convert error. So I did it all one more time to gather the log. That .kinf2018 keeps recreating. I don't know if that's expected or not.

image

Calibre Log
calibre Debug log
calibre 4.23  embedded-python: True is64bit: True
Darwin-20.1.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '20.1.0', 'Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:10 PDT 2020; root:xnu-7195.50.7~2/RELEASE_ARM64_T8101')
Python 2.7.16
OSX: ('10.16', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: DeDRM (6, 8, 0)
calibre 4.23  embedded-python: True is64bit: True
Darwin-20.1.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '20.1.0', 'Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:10 PDT 2020; root:xnu-7195.50.7~2/RELEASE_ARM64_T8101')
Python 2.7.16
OSX: ('10.16', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: DeDRM (6, 8, 0)
devicePixelRatio: 1.0
logicalDpi: 72.0 x 72.0
physicalDpi: 92.0000013817 x 91.0000013667
Using calibre Qt style: True
[0.00] Starting up...
[0.02] Showing splash screen...
[0.21] splash screen shown
[0.21] Initializing db...
[0.23] db initialized
[0.23] Constructing main UI...
[0.81] main UI initialized...
[0.81] Hiding splash screen
[4.29] splash screen hidden
[4.29] Started up in 4.29 seconds with 255 books
DeDRM v6.8.0: Trying to decrypt B083J7QQ89_EBOK.azw
Using Library AlfCrypto Python
MobiDeDrm v0.42.
Copyright © 2008-2017 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: The Cipher (Nina Guerrera)
Found 0 keys to try after 0.0 seconds
Crypto Type is: 2
DeDRM v6.8.0: Failed to decrypt with error: No key found in 0 keys tried.
DeDRM v6.8.0: Looking for new default Kindle Key after 0.0 seconds
Found k4Mac kinf2018 file: /Users/lisa/Library/Application Support/Kindle/storage/.kinf2018
Couldn't decrypt file.
DeDRM v6.8.0: Ultimately failed to decrypt after 0.2 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md
Running file type plugin DeDRM failed with traceback:
Traceback (most recent call last):
  File "site-packages/calibre/customize/ui.py", line 172, in _run_filetype_plugins
  File "calibre_plugins.dedrm.__init__", line 625, in run
  File "calibre_plugins.dedrm.__init__", line 575, in KindleMobiDecrypt
DeDRMError: DeDRM v6.8.0: Ultimately failed to decrypt after 0.2 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md
Worker Launch took: 0.0827639102936
Job: 0 Convert book 1 of 1 (The Cipher (Nina Guerrera)) finished
Convert book 1 of 1 (The Cipher (Nina Guerrera))
	Conversion options changed from defaults:
	  read_metadata_from_opf: u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_mmt1ZU/hFtlGT.opf'
	  verbose: 2
	  cover: u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_mmt1ZU/krqMvT.jpeg'
	  input_profile: u'kindle'
	  output_profile: u'kindle_oasis'
	Resolved conversion options
	calibre version: 4.23.0
	{'asciiize': False,
	 'author_sort': None,
	 'authors': None,
	 'base_font_size': 0.0,
	 'book_producer': None,
	 'change_justification': u'original',
	 'chapter': u"//*[((name()='h1' or name()='h2') and re:test(., '\\s*((chapter|book|section|part)\\s+)|((prolog|prologue|epilogue)(\\s+|$))', 'i')) or @class = 'chapter']",
	 'chapter_mark': u'pagebreak',
	 'comments': None,
	 'cover': u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_mmt1ZU/krqMvT.jpeg',
	 'debug_pipeline': None,
	 'dehyphenate': True,
	 'delete_blank_paragraphs': True,
	 'disable_font_rescaling': False,
	 'dont_split_on_page_breaks': False,
	 'duplicate_links_in_toc': False,
	 'embed_all_fonts': False,
	 'embed_font_family': None,
	 'enable_heuristics': False,
	 'epub_flatten': False,
	 'epub_inline_toc': False,
	 'epub_toc_at_end': False,
	 'epub_version': u'2',
	 'expand_css': False,
	 'extra_css': None,
	 'extract_to': None,
	 'filter_css': u'',
	 'fix_indents': True,
	 'flow_size': 260,
	 'font_size_mapping': None,
	 'format_scene_breaks': True,
	 'html_unwrap_factor': 0.4,
	 'input_encoding': None,
	 'input_profile': <calibre.customize.profiles.KindleInput object at 0x117df5090>,
	 'insert_blank_line': False,
	 'insert_blank_line_size': 0.5,
	 'insert_metadata': False,
	 'isbn': None,
	 'italicize_common_cases': True,
	 'keep_ligatures': False,
	 'language': None,
	 'level1_toc': None,
	 'level2_toc': None,
	 'level3_toc': None,
	 'line_height': 0.0,
	 'linearize_tables': False,
	 'margin_bottom': 5.0,
	 'margin_left': 5.0,
	 'margin_right': 5.0,
	 'margin_top': 5.0,
	 'markup_chapter_headings': True,
	 'max_toc_links': 50,
	 'minimum_line_height': 120.0,
	 'no_chapters_in_toc': False,
	 'no_default_epub_cover': False,
	 'no_inline_navbars': False,
	 'no_svg_cover': False,
	 'output_profile': <calibre.customize.profiles.KindleOasisOutput object at 0x117df5690>,
	 'page_breaks_before': u"//*[name()='h1' or name()='h2']",
	 'prefer_metadata_cover': False,
	 'preserve_cover_aspect_ratio': False,
	 'pretty_print': True,
	 'pubdate': None,
	 'publisher': None,
	 'rating': None,
	 'read_metadata_from_opf': u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_mmt1ZU/hFtlGT.opf',
	 'remove_fake_margins': True,
	 'remove_first_image': False,
	 'remove_paragraph_spacing': False,
	 'remove_paragraph_spacing_indent_size': 1.5,
	 'renumber_headings': True,
	 'replace_scene_breaks': u'',
	 'search_replace': '[]',
	 'series': None,
	 'series_index': None,
	 'smarten_punctuation': False,
	 'sr1_replace': None,
	 'sr1_search': None,
	 'sr2_replace': None,
	 'sr2_search': None,
	 'sr3_replace': None,
	 'sr3_search': None,
	 'start_reading_at': None,
	 'subset_embedded_fonts': False,
	 'tags': None,
	 'timestamp': None,
	 'title': None,
	 'title_sort': None,
	 'toc_filter': None,
	 'toc_threshold': 6,
	 'toc_title': None,
	 'transform_css_rules': '[]',
	 'unsmarten_punctuation': False,
	 'unwrap_lines': True,
	 'use_auto_toc': False,
	 'verbose': 2}
	InputFormatPlugin: MOBI Input running
	on /var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_mmt1ZU/g4FRD3.azw
	Found KF8 MOBI of type u'standalone'
	Python function terminated unexpectedly: The Cipher (Nina Guerrera)
	Traceback (most recent call last):
	  File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 187, in main
	    return run_entry_point()
	  File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 121, in run_entry_point
	    return getattr(pmod, func)()
	  File "site-packages/calibre/utils/ipc/worker.py", line 209, in main
	  File "site-packages/calibre/gui2/convert/gui_conversion.py", line 43, in gui_convert_override
	  File "site-packages/calibre/gui2/convert/gui_conversion.py", line 28, in gui_convert
	  File "site-packages/calibre/ebooks/conversion/plumber.py", line 1110, in run
	  File "site-packages/calibre/customize/conversion.py", line 246, in __call__
	  File "site-packages/calibre/ebooks/conversion/plugins/mobi_input.py", line 47, in convert
	  File "site-packages/calibre/ebooks/mobi/reader/mobi8.py", line 86, in __call__
	  File "site-packages/calibre/ebooks/mobi/reader/mobi6.py", line 158, in check_for_drm
	calibre.ebooks.DRMError: The Cipher (Nina Guerrera)

@ElleKayEm
Copy link

And you redownloaded the book after this new installation? And added the fresh copy to calibre?

@ghost
Copy link
Author

ghost commented Nov 20, 2020

Yes, I deleted from Calibre, deleted from My Kindle Library and redownloaded... multiple times.

@ghost
Copy link
Author

ghost commented Nov 20, 2020

I also have all these same versions installed on a MacBook running Catalina, and it all works fine. I'm suspicious that it has something to do with Big Sur.

@ElleKayEm
Copy link

Do you have KFX downloads disabled?

@ghost
Copy link
Author

ghost commented Nov 20, 2020

I think so. If you mean, running:
chmod -x /Applications/Kindle.app/Contents/MacOS/renderer-test

Then yes. If not, then probably not.

@ElleKayEm
Copy link

Yes, that's what I meant. I don't know what else to suggest. Seems like it might be something to do with Big Sur. But I have no idea.

@ghost
Copy link
Author

ghost commented Nov 21, 2020

Are you one of the developers? If not, are they monitoring this thread? More and more users will be upgrading to Big Sur. Seems like this might be something they'd want to look into. I'm willing to try any other ideas to help debug this including (but not limited to), uninstalling, reinstalling, changing up the order of steps, etc.

@ElleKayEm
Copy link

I'm not. As you say, I'm sure there will be more reports if Big Sur is the issue, and @apprenticeharper and company will become aware. I will also ask around myself.

@ghost
Copy link
Author

ghost commented Nov 21, 2020

I tried posting to mobileread, but they closed it immediately. I didn't even post instructions, I just spoke about the problem in the abstract. I don't know where else to post to see if others are having the same issue.

@ElleKayEm
Copy link

Yeah, I think they were a little overzealous with that. Hopefully we get some responses from the other thread there.

@ghost
Copy link
Author

ghost commented Nov 21, 2020

image
I decided to look at the customization of deDRM and the Keys are blank! Gonna try copying them from my Catalina Mac...

@ghost
Copy link
Author

ghost commented Nov 21, 2020

Saving the keys from my MacBook running Catalina and importing them to my Mac mini running Big Sur did not work:
image
I still received the "this book is locked by DRM" error.

If I click the plus sign, I get this error:
image

So the problem seems to lie with the non-existence of the proper keys. Maybe @apprenticeharper can offer some help?

@ElleKayEm
Copy link

DeDRM will only automatically save a key that successfully decrypted a book. So that's why nothing was there at first. I think the error message you got when trying to add the key may be because DeDRM is not able to decrypt the key, not because it can't find one at all. At least that's what the log from importing a book seemed to indicate. And the key for each installation of the Kindle is unique (even a reinstallation on the same machine which requires you to login again can change the key). So that's why the keys from your Catalina don't work on books from the Big Sur Kindle app. I'd be curious to see if you could decrypt a book from the Catalina Kindle app with the Big Sur calibre/DeDRM setup with the Catalina's imported keys.

@ghost
Copy link
Author

ghost commented Nov 21, 2020

Ha! So that worked.
I downloaded the book on my Catalina MacBook in Kindle for Mac. Then I copied that .azw file from the Catalina Mac to my Big Sur Mac. (Big Sur Mac had the keys from the Catalina Mac.) Catalina did it's import and it removed the DRM and created the .azw3 (and .epub) files!

So how do we get the proper keys for Big Sur???

@ElleKayEm
Copy link

Therein lies the problem. It appears that DeDRM can't decrypt the key for the Big Sur installation of the Kindle app. It's beyond my knowledge of this to see why. But at least that might narrow the issue for those who can figure it out.

@ghost
Copy link
Author

ghost commented Nov 22, 2020

If I use the "download via usb" method, it all works fine! TBH, that's easier anyway.

@ElleKayEm
Copy link

Ah, didn't know you had an E-ink Kindle. This will still need looking at, but at least book decryption seems fine with Big Sur. It's just getting and decrypting the key for Kindle for Mac that is giving trouble.

@ghost
Copy link
Author

ghost commented Nov 22, 2020

Thanks for all your help. If and when someone has time to fix this, just @ me or something and I'll be happy to test it.

@Augilare
Copy link

Patch kindle for mac catalina and Big Sur with following command,
chmod -x /Applications/Kindle.app/Contents/MacOS/renderer-test
OR
sudo chmod -x /Applications/Kindle.app/Contents/MacOS/renderer-test,
Then right-click kindle book to download. DON'T double click or open it before the drm removing.
Convert Kindle KFX to EPUB/PDF Free on macOS Big Sur (Effective)

@net-buoy
Copy link

net-buoy commented Nov 25, 2020 via email

@ElleKayEm
Copy link

ElleKayEm commented Nov 25, 2020

@Augilare
The OP did this. The problem here is decrypting the Kindle for Mac key. Not KFX format (which can be decrypted if from K4Mac 1.26 anyway).

@ZolaLa9
Copy link
Contributor

ZolaLa9 commented Nov 25, 2020

@Lisa10101 — have you tried this with K4Mac 1.30?

The blogger referenced by @Augilare reports success using Calibre 4.23, DeDRM 6.8, K4Mac 1.30 and Mac mini M1 Big Sur.

@ghost
Copy link
Author

ghost commented Nov 25, 2020

@ZolaLa9 I have not tried 1.30, mostly because DeDRM 6.8 states it's not compatible with anything over 1.26. If I give it a try later, I'll report back.

@ElleKayEm
Copy link

@Lisa10101
That is in regard to KFX format. If you have KFX downloads disabled (chmod -x ...), you can use 1.30.

@ZolaLa9
Copy link
Contributor

ZolaLa9 commented Nov 25, 2020

@Lisa10101 — This post is updated regularly and tends to have the most up-to-date info on K4Mac compatibility: https://www.mobileread.com/forums/showpost.php?p=3819708&postcount=508

Not sure how active you are on MobileRead, but there is also active discussion happening here on DeDRM 7.0 beta and Mac: https://www.mobileread.com/forums/showthread.php?s=32fb948c4aa246ac91444f4a00ca6632&p=4061946#post4061946

@ghost
Copy link
Author

ghost commented Nov 25, 2020

I'm not interested in installing things that are "beta" or "in development", so I felt this was the better place to look for compatibility between DeDRM and Calibre:
https://apprenticealf.wordpress.com
It was just updated today, so I was going off the previous post which says this:

This means that this release now works with version 1.26 and earlier of Kindle for PC and Kindle for Mac.

and this:

Thanks to work by several people, the DeDRM calibre plugin can decrypt KFX format ebooks from Amazon as long as they are downloaded with Kindle for Mac/PC prior to version 1.27.

@ghost
Copy link
Author

ghost commented Nov 25, 2020

@ZolaLa9 @Augilare Just tried it using Kindle for Mac 1.30 (and yes, I did the step to disable KFX before downloading the book) and it did not work:
image

I'm pretty skeptical that the author of that blog actually tested this on a new Mac mini M1 running Big Sur.

@ElleKayEm
Copy link

And the next sentence:

Books which require the later versions of Kindle for PC/Mac will not be able to be decrypted at this time.

But non-KFX formats can be decrypted from the later versions.

@ghost
Copy link
Author

ghost commented Nov 25, 2020

@ElleKayEm It doesn't say "But non-KFX formats can be decrypted from the later versions."
There is nothing in that post from June 18, 2020 that would lead one to believe you could use DeDRM 6.8 with Kindle for Mac 1.30.
Regardless... if all or most of your books are KFX formats, then anything over 1.26 does not work. (I don't want to muddy the waters here by discussing other tests.. but before I switched to Mac mini M1 Big Sur, I was using a MacBook Catalina and I was unable to use 1.30 there (disabling KFX of course) with DeDRM 6.8. I had to downgrade to 1.26 and disable KFX for it to work. )

@ElleKayEm
Copy link

Have a look at this old issue #255. It seemed to be a similar issue with DeDRM not being able to decrypt the Kindle for Mac key. Sounds like the key encryption is tied to an ID number from the internal drive. Perhaps something with Big Sur or your particular setup is causing DeDRM to grab the wrong number.

(DeDRM 6.8 addressed two things: decryption of the new Kindle for PC/Mac key which first appeared in version 1.25 and new KFX DRM used in versions 1.25-1.26. A further change to KFX DRM happened beginning with 1.27 which DeDRM does not handle yet. But the DRM on the older formats that you get when you disable KFX format downloads has not changed.)

@j-howell
Copy link

DeDRM uses a variety of cryptographic routines that do much of the same thing due to its history as a bunch of separate tools created by different individuals. I believe that the problem with Big Sur is caused by one of those no longer being available or having its named changed. This and others like it could be eliminated in favor of the cryptography library built into calibre.

@protochron
Copy link
Contributor

protochron commented Nov 26, 2020

It looks like the plugin is failing to load libcrypto because of a change in the way that Big Sur handles loading dynamic libraries. The libcrypto.dylib file no longer exists on disk, so if the plugin is running on a Mac then the code in https://github.com/apprenticeharper/DeDRM_tools/blob/v6.8.0/DeDRM_plugin/kindlekey.py#L1184-L1186 needs to try loading /usr/lib/libcrypto.dylib directly when libcrypto is None and handling an exception if it doesn't work after that. I tested this with Big Sur, Kindle for Mac 1.30, DeDRM 6.8.0 and Calibre 4.21 and it was able to handle getting the key just fine.

See saltstack/salt#57787 for more info re: failing to load dylibs in Big Sur.

It does seem like loading Calibre's version of OpenSSL would solve this once and for all, but I'm happy to put in a PR with the fix.

@ZolaLa9
Copy link
Contributor

ZolaLa9 commented Nov 26, 2020

@protochronAWESOME!!! If you would put in a PR, it would be mucho appreciated!

@ZolaLa9
Copy link
Contributor

ZolaLa9 commented Nov 26, 2020

before I switched to Mac mini M1 Big Sur, I was using a MacBook Catalina and I was unable to use 1.30 there (disabling KFX of course) with DeDRM 6.8. I had to downgrade to 1.26 and disable KFX for it to work.

@Lisa10101 — Catalina + K4Mac 1.30 + Calibre 4.23 + Alf 6.8.0 = working for me

@apprenticeharper
Copy link
Owner

I don't have Big Sur, and I'm still working on other Python 3 problems. A pull request that fixed this problem would be much appreciated.

protochron added a commit to protochron/DeDRM_tools that referenced this issue Nov 28, 2020
It looks like Big Sur removed `libcrypto.dylib` as a file on the
filesystem, so loading it using `ctypes.find_library` fails which breaks
Kindle decryption. Now to load a dylib you need to attempt to load it
directly and the operating system will load the dylib from the OS' cache
or fail.

This fixes the problem by explicitly setting the path to libcrypto to
`/usr/lib/libcrypto.dylib` if `ctypes.find_library` does not find the
file, loading the dylib and raising an exception if it fails at that
point.

See saltstack/salt#5778 for more detailed info.

Closes apprenticeharper#1369.
protochron added a commit to protochron/DeDRM_tools that referenced this issue Nov 28, 2020
It looks like Big Sur removed `libcrypto.dylib` as a file on the
filesystem, so loading it using `ctypes.find_library` fails which breaks
Kindle decryption. Now to load a dylib you need to attempt to load it
directly and the operating system will load the dylib from the OS' cache
or fail.

This fixes the problem by explicitly setting the path to libcrypto to
`/usr/lib/libcrypto.dylib` if `ctypes.find_library` does not find the
file, loading the dylib and raising an exception if it fails at that
point.

See saltstack/salt#5778 for more detailed info.

Closes apprenticeharper#1369.
protochron added a commit to protochron/DeDRM_tools that referenced this issue Nov 28, 2020
It looks like Big Sur removed `libcrypto.dylib` as a file on the
filesystem, so loading it using `ctypes.find_library` fails which breaks
Kindle decryption. Now to load a dylib you need to attempt to load it
directly and the operating system will load the dylib from the OS' cache
or fail.

This fixes the problem by explicitly setting the path to libcrypto to
`/usr/lib/libcrypto.dylib` if `ctypes.find_library` does not find the
file, loading the dylib and raising an exception if it fails at that
point.

See saltstack/salt#5778 for more detailed info.

Closes apprenticeharper#1369.
@jcs
Copy link

jcs commented Nov 28, 2020

#1381 fixes the key finding/loading for me on Big Sur, though it still fails importing an azw file:

DeDRM v7.0.0: Trying to decrypt book.azw
Using Library AlfCrypto Python
Using Library AlfCrypto Python
MobiDeDrm v1.0.
Copyright © 2008-2020 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: [title]
Got SerialNumber from database default_key
Got UsernameHash from database default_key
Found 4 keys to try after 0.0 seconds
Crypto Type is: 2
DeDRM v7.0.0: Failed to decrypt with error: No key found in 4 keys tried.
DeDRM v7.0.0: Looking for new default Kindle Key after 0.0 seconds
Found k4Mac kinf2018 file: /Users/jcs/Library/Containers/com.amazon.Kindle/Data/Library/Application Support/Kindle/storage/.kinf2018
Found k4Mac kinf2018 file: /Users/jcs/Library/Application Support/Kindle/storage/.kinf2018
trying username  b'jcs'  on file  /Users/jcs/Library/Containers/com.amazon.Kindle/Data/Library/Application Support/Kindle/storage/.kinf2018
trying IDString: b'bb8ce...'
trying IDString: b'bb8ce...'
trying IDString: b'bd9b4...'
Decrypted key file using IDString 'bd9b4...' and UserName 'jcs'
trying username  b'jcs'  on file  /Users/jcs/Library/Application Support/Kindle/storage/.kinf2018
trying IDString: b'bb8ce...'
trying IDString: b'bb8ce...'
trying IDString: b'bd9b4...'
Decrypted key file using IDString 'bd9b4...' and UserName 'jcs'
DeDRM v7.0.0: Found 1 new key
MobiDeDrm v1.0.
Copyright © 2008-2020 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: [title]
Got SerialNumber from database default_key_2
Got UsernameHash from database default_key_2
Found 4 keys to try after 2.3 seconds
Crypto Type is: 2
DeDRM v7.0.0: Ultimately failed to decrypt after 2.3 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md
Running file type plugin DeDRM failed with traceback:
Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 533, in KindleMobiDecrypt
    book = k4mobidedrm.GetDecryptedBook(path_to_ebook,kindleDatabases,androidFiles,serials,pids,self.starttime)
  File "calibre_plugins.dedrm.k4mobidedrm", line 238, in GetDecryptedBook
    mb.processBook(totalpids)
  File "calibre_plugins.dedrm.mobidedrm", line 476, in processBook
    raise DrmException("No key found in {0:d} keys tried.".format(len(goodpids)))
calibre_plugins.dedrm.mobidedrm.DrmException: No key found in 4 keys tried.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre/customize/ui.py", line 172, in _run_filetype_plugins
  File "calibre_plugins.dedrm.__init__", line 626, in run
    decrypted_ebook = self.KindleMobiDecrypt(path_to_ebook)
  File "calibre_plugins.dedrm.__init__", line 576, in KindleMobiDecrypt
    raise DeDRMError("{0} v{1}: Ultimately failed to decrypt after {2:.1f} seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md".format(PLUGIN_NAME, PLUGIN_VERSION,time.time()-self.starttime))
calibre_plugins.dedrm.DeDRMError: DeDRM v7.0.0: Ultimately failed to decrypt after 2.3 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md

@TheFreediver
Copy link

I know this is an old thread but I had a similar issue with Catalina, kindle 1.26 and calibre 4 and 5. What Aguilare said not to actually click the book when downloading, helped me. It appears as soon as kindle opens the book it says preparing your book and deletes the azw format. Despite using kindle 1.26, updates disabled, updates folder removed, etc.
So conclusion don't open the book in kindle app on mac os until you have imported in calibre.

Then right-click kindle book to download. DON'T double click or open it before the drm removing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants