search mode triggers email parsing error

[pazz]

I get a popup and corresponding error log AttributeError: 'str' object has no attribute 'is_attachment'
when refreshing "down the search set" in search mode.

Software Versions

• Python version:
• Alot version: master

To Reproduce

1. search for *
2. scroll down a few pages using pgdown
3. hit @ for refresh.

Error Log

 File "/home/pazz/projects/alot/alot/ui.py", line 723, in apply_command
    cmd.apply(self)
  File "/home/pazz/projects/alot/alot/commands/globals.py", line 172, in apply
    ui.update()
  File "/home/pazz/projects/alot/alot/ui.py", line 668, in update
    self.mainloop.draw_screen()
  File "/usr/lib/python3/dist-packages/urwid/main_loop.py", line 586, in draw_screen
    canvas = self._topmost_widget.render(self.screen_size, focus=True)
  File "/usr/lib/python3/dist-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/lib/python3/dist-packages/urwid/decoration.py", line 226, in render
    canv = self._original_widget.render(size, focus=focus)
  File "/usr/lib/python3/dist-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/lib/python3/dist-packages/urwid/container.py", line 1085, in render
    body = self.body.render((maxcol, maxrow-ftrim-htrim),
  File "/home/pazz/projects/alot/alot/buffers/buffer.py", line 19, in render
    return self.body.render(size, focus)
  File "/usr/lib/python3/dist-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/lib/python3/dist-packages/urwid/listbox.py", line 470, in render
    middle, top, bottom = self.calculate_visible(
  File "/usr/lib/python3/dist-packages/urwid/listbox.py", line 416, in calculate_visible
    next, pos = self._body.get_next( pos )
  File "/home/pazz/projects/alot/alot/walker.py", line 46, in get_next
    return self._get_at_pos(start_from + self.direction)
  File "/home/pazz/projects/alot/alot/walker.py", line 72, in _get_at_pos
    widget = self._get_next_item()
  File "/home/pazz/projects/alot/alot/walker.py", line 85, in _get_next_item
    next_widget = self.containerclass(next_obj, **self.kwargs)
  File "/home/pazz/projects/alot/alot/widgets/search.py", line 26, in __init__
    self.rebuild()
  File "/home/pazz/projects/alot/alot/widgets/search.py", line 60, in rebuild
    width, part = build_text_part(partname, self.thread,
  File "/home/pazz/projects/alot/alot/widgets/search.py", line 145, in build_text_part
    content = prepare_string(name, thread, maxw)
  File "/home/pazz/projects/alot/alot/widgets/search.py", line 213, in prepare_string
    s = content(thread)
  File "/home/pazz/projects/alot/alot/widgets/search.py", line 178, in prepare_authors_string
    return thread.get_authors_string() or '(None)'
  File "/home/pazz/projects/alot/alot/db/thread.py", line 195, in get_authors_string
    for aname, aaddress in self.get_authors():
  File "/home/pazz/projects/alot/alot/db/thread.py", line 158, in get_authors
    msgs = sorted(self.get_messages().keys(),
  File "/home/pazz/projects/alot/alot/db/thread.py", line 247, in get_messages
    self._toplevel_messages.append(accumulate(self._messages, m))
  File "/home/pazz/projects/alot/alot/db/thread.py", line 237, in accumulate
    M = Message(self._dbman, msg, thread=self)
  File "/home/pazz/projects/alot/alot/db/message.py", line 71, in __init__
    self.mime_part = get_body_part(self.get_email())
  File "/home/pazz/projects/alot/alot/db/utils.py", line 491, in get_body_part
    body_part = mail.get_body(preferencelist)
  File "/usr/lib/python3.8/email/message.py", line 1018, in get_body
    for prio, part in self._find_body(self, preferencelist):
  File "/usr/lib/python3.8/email/message.py", line 989, in _find_body
    yield from self._find_body(subpart, preferencelist)
  File "/usr/lib/python3.8/email/message.py", line 989, in _find_body
    yield from self._find_body(subpart, preferencelist)
  File "/usr/lib/python3.8/email/message.py", line 978, in _find_body
    if part.is_attachment():
AttributeError: 'str' object has no attribute 'is_attachment'

[pazz]

Bumped into this again just now. Is anyone else gitting this ? It looks like a bug in the python email module!

[pazz]

https://gist.github.com/pazz/953e0f7cce8220c59d742aecc8779bb7

https://bugs.python.org/issue41010

[josch]

Hi,

I'm getting this traceback on some spam mail, which I guess is the same problem?

Traceback (most recent call last):
  File "/usr/bin/alot", line 11, in <module>
    load_entry_point('alot==0.9.1', 'console_scripts', 'alot')()
  File "/usr/share/alot/alot/__main__.py", line 137, in main
    UI(dbman, cmdstring)
  File "/usr/share/alot/alot/ui.py", line 146, in __init__
    self.mainloop.run()
  File "/usr/lib/python3/dist-packages/urwid/main_loop.py", line 287, in run
    self._run()
  File "/usr/lib/python3/dist-packages/urwid/main_loop.py", line 385, in _run
    self.event_loop.run()
  File "/usr/lib/python3/dist-packages/urwid/main_loop.py", line 1346, in run
    reraise(*exc_info)
  File "/usr/lib/python3/dist-packages/urwid/compat.py", line 58, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/urwid/main_loop.py", line 1360, in wrapper
    rval = f(*args,**kargs)
  File "/usr/lib/python3/dist-packages/urwid/main_loop.py", line 173, in cb
    callback(self, user_data)
  File "/usr/share/alot/alot/ui.py", line 627, in clear
    self.clear_notify(msgs)
  File "/usr/share/alot/alot/ui.py", line 524, in clear_notify
    self.update()
  File "/usr/share/alot/alot/ui.py", line 668, in update
    self.mainloop.draw_screen()
  File "/usr/lib/python3/dist-packages/urwid/main_loop.py", line 588, in draw_screen
    canvas = self._topmost_widget.render(self.screen_size, focus=True)
  File "/usr/lib/python3/dist-packages/urwid/widget.py", line 145, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/lib/python3/dist-packages/urwid/decoration.py", line 226, in render
    canv = self._original_widget.render(size, focus=focus)
  File "/usr/lib/python3/dist-packages/urwid/widget.py", line 145, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/lib/python3/dist-packages/urwid/container.py", line 1089, in render
    body = self.body.render((maxcol, maxrow-ftrim-htrim),
  File "/usr/share/alot/alot/buffers/buffer.py", line 19, in render
    return self.body.render(size, focus)
  File "/usr/lib/python3/dist-packages/urwid/widget.py", line 145, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/lib/python3/dist-packages/urwid/listbox.py", line 439, in render
    middle, top, bottom = self.calculate_visible(
  File "/usr/lib/python3/dist-packages/urwid/listbox.py", line 385, in calculate_visible
    next, pos = self._body.get_next( pos )
  File "/usr/share/alot/alot/walker.py", line 46, in get_next
    return self._get_at_pos(start_from + self.direction)
  File "/usr/share/alot/alot/walker.py", line 72, in _get_at_pos
    widget = self._get_next_item()
  File "/usr/share/alot/alot/walker.py", line 85, in _get_next_item
    next_widget = self.containerclass(next_obj, **self.kwargs)
  File "/usr/share/alot/alot/widgets/search.py", line 26, in __init__
    self.rebuild()
  File "/usr/share/alot/alot/widgets/search.py", line 60, in rebuild
    width, part = build_text_part(partname, self.thread,
  File "/usr/share/alot/alot/widgets/search.py", line 145, in build_text_part
    content = prepare_string(name, thread, maxw)
  File "/usr/share/alot/alot/widgets/search.py", line 213, in prepare_string
    s = content(thread)
  File "/usr/share/alot/alot/widgets/search.py", line 178, in prepare_authors_string
    return thread.get_authors_string() or '(None)'
  File "/usr/share/alot/alot/db/thread.py", line 195, in get_authors_string
    for aname, aaddress in self.get_authors():
  File "/usr/share/alot/alot/db/thread.py", line 158, in get_authors
    msgs = sorted(self.get_messages().keys(),
  File "/usr/share/alot/alot/db/thread.py", line 247, in get_messages
    self._toplevel_messages.append(accumulate(self._messages, m))
  File "/usr/share/alot/alot/db/thread.py", line 237, in accumulate
    M = Message(self._dbman, msg, thread=self)
  File "/usr/share/alot/alot/db/message.py", line 71, in __init__
    self.mime_part = get_body_part(self.get_email())
  File "/usr/share/alot/alot/db/utils.py", line 487, in get_body_part
    body_part = mail.get_body(preferencelist)
  File "/usr/lib/python3.9/email/message.py", line 1018, in get_body
    for prio, part in self._find_body(self, preferencelist):
  File "/usr/lib/python3.9/email/message.py", line 989, in _find_body
    yield from self._find_body(subpart, preferencelist)
  File "/usr/lib/python3.9/email/message.py", line 978, in _find_body
    if part.is_attachment():
AttributeError: 'str' object has no attribute 'is_attachment'

Glad to hear that you identified where this is coming from! The bad part is, that due to this exception, the mails do not show up in the alot search, so I currently am marking them as spam manually with "notmuch tag +spam". In case it matters, here are some more spam mails that triggered this problem for me:

https://gist.github.com/josch/ff8ac2db97dba1af49d5c3e4bf8d063a

[josch]

And another one (gzipped):

mail.gz