Complete tasks with end value, add state and closed UDAs to GitHub

[pawamoy]

Issue #620

When pulling closed issues (with custom query), bugwarrior will use the closed_at field from GitHub to set the end value of the task and mark it as "done".

Two new UDAs in this PR:

• githubclosedon (to be consistent with githubcreatedon): date
• githubstate: string, which is the state of the issue/PR (open, closed or merged)

To implement this for other services, one will need:

1. to add the required UDAs (esp. closing date) in the given service
2. to update the to_taskwarrior method of the service to add the end item in the returned dictionary

With the end attribute set and true (a valid date), bugwarrior will automatically complete the task after adding/modifying it. I don't know how taskwarrior itself reacts when asked to add a new task with an end value already set, but I tried this code and had no issue with it. Tasks were successfully created or modified and immediately after, completed.

TODO:

• update docs

[pawamoy]

Python 2 jobs fixed by #618

[wookietreiber]

I'm wondering if that error in travis is spurious or not:

Failure: ImportError (No module named configparser) ... ERROR

It only effects Python 2.7 ...

[pawamoy]

Yeah I don't know why it appears only now and not in previous builds. PR #618 fixes it though.

[wookietreiber]

#618 is merged, please rebase on latest develop, then the checks should all pass.

[wookietreiber]

LGTM 👍

I suppose you tested this locally: does this also create completed tasks from closed issues or is it just closing issues that were created by bugwarrior before? I would really like this for my burndown charts :)

[pawamoy]

I would really like this for my burndown charts :)

This is exactly why I opened this PR 😀 !

Yes, it will create completed tasks from closed issues, but (and I'm not targeting you particularly but writing for everyone instead) you have to understand two things:

1. how bugwarrior closes tasks by default: the default behavior is that bugwarrior pulls only open issues. In subsequent pulls, each and every bugwarrior task that is missing from the pull is closed. Something like:

pull A B C from service
create A B C in taskwarrior
close B online
pull A C from service
  B is missing -> complete B

2. this PR adds a new behavior: if the pulled issue has an end, then bugwarrior will complete it after adding (it did not exist before) or modifying it (it was already pulled before). But that means you have to pull closed issues, and you can only do this by writing a custom SERVICE.query. For GitHub, that would be something like github.query = involves:pawamoy archived:false. Note the absence of state:open in the query. A more concrete example would be:

[github_pawamoy]
service = github

github.login = pawamoy
github.token = <token>

github.include_user_issues = True  # include the issues I opened
github.include_user_repos = True   # include the issues of my repositories
github.involved_issues = False     # this is overwritten with a custom query to include closed issues

github.query = involves:pawamoy archived:false  # I'm not interested in archived issues
# I could also have played with author: assignee: and mentions:
# Try it online before, to see what works and what doesn't

With the example in the second point above, a few questions come to mind:

• Does github.include_user_issues = True include closed issues I opened? I don't think so, but this is fixed with the custom query.
• Does github.include_user_repos = True include closed issues opened by others on my repositories? I don't think so, and this is a problem because even the custom query could miss some of them (issue opened and closed by others than me, without me commenting, assigning to it, or being mentioned).

So, my conclusion is that bugwarrior should offer a way to include closed issues. A simple flag like SERVICE.include_closed_issues would do. Of course the service in question must be able to tell the issue is closed (and when) to set the end value. Otherwise bugwarrior creates those closed issues as normal, non-completed tasks.

I'm gonna add this to my PR in the docs somewhere, maybe also in a contributing section 🙂

Other remark:

• the taskwarrior Python API does not provide a task_log method (like the taskwarrior's log command), so a newly-pulled, closed issue will first be added, then completed. I don't know how taskwarrior itself reacts when asked to add a new task with an end value already set, but I tried this code and had no issue with it.

[wookietreiber]

Both include_user_repos and include_user_issues fetch only open issues. To change this, you would need to add state=all to their respective queries, i.e. here and here.

[pawamoy]

Thanks, we could start by implementing this for GitHub indeed 🙂 In another PR though.

[pawamoy]

Added docs for the two new UDAs. Ready to merge @wookietreiber @ralphbean !

[ryneeverett]

Is there a reason we're calling this end and not closed?

[pawamoy]

Yes! For the same reason created is named entry I guess: because this dictionary is ultimately passed to taskw, the Python API module for taskwarrior, and taskwarrior's end date field is named end.

The code path to show it:

• command.pull
  • create issue_generator with:
    • service.aggregate_issues
      • service._aggregate_issues
        • SERVICE.issues
          • return issues from records (get_issue_for_record)
          • records are obtained via get_taskwarrior_record
          • get_taskwarrior_record calls to_taskwarrior (our interest here)
  • db.synchronize(issue_generator, ...)
    • for issue in issue_generator:
      • issue_dict = dict(issue)
        • task.add(**issue_dict): Documentation contains "Takes any of the keywords allowed by taskwarrior like proj or prior."