Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reconcile_transaction() somehow breaks the "rules" functionality on actual #68

Closed
2 tasks done
tcpr1 opened this issue Sep 8, 2024 · 10 comments · Fixed by #85
Closed
2 tasks done

reconcile_transaction() somehow breaks the "rules" functionality on actual #68

tcpr1 opened this issue Sep 8, 2024 · 10 comments · Fixed by #85
Labels
bug Something isn't working

Comments

@tcpr1
Copy link

tcpr1 commented Sep 8, 2024

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of actualpy.

Reproducible example

from actual import Actual
from actual.queries import get_accounts, get_transactions, reconcile_transaction

csv_data = [["Date","Payee","Notes","Category","Amount","Cleared", "imported_ID"]]

with Actual(base_url=URL_ACTUAL, password=PASSWORD_ACTUAL, file=FILE_ACTUAL) as actual:

accounts = get_accounts(actual.session)
for account in accounts:

     for row in csv_data:
        # here, we define the basic information from the file
        date, payee, notes, category, amount, cleared, imported_ID = (
            datetime.strptime(row[0], "%Y-%m-%d").date(),  # transform to date
            row[1],
            row[2],
            row[3],
            decimal.Decimal(row[4]),
            row[5] == "Cleared",
            row[6],
        )
        t = reconcile_transaction(
            actual_session,
            date,
            account,
            payee,
            notes,
            category,
            amount,
            imported_ID,
            cleared=cleared,
            imported_payee=payee,
            already_matched=added_transactions,
        )
        added_transactions.append(t)
actual.commit()

Log output

No response

Issue description

I am using reconcile_transaction() to import bank data into Actual.
Everything seems to be working fine, but the "rules" funcionality to rename payees do not work.

One thing I also noticed is when I import .ofx or .csv inside Actual, the payee name gets bolded, but it is not the case for the payees names of imported transactions through this API.

Expected behavior

After actual.commit() the newly created transactions should be renamed based on Actual rules automatically.

Installed versions

  • actualpy version: 0.4.0 - Actual Server version: 24.9.0
@tcpr1 tcpr1 added the bug Something isn't working label Sep 8, 2024
@bvanelli
Copy link
Owner

bvanelli commented Sep 8, 2024

I understand your use case, maybe adding a run_rules on commit would be enough, but it's not optimal because this API tries to mimic adding entries via frontend, and if you add a transaction via frontend, no rule will be run (as far as I could test).

Wouldn't something like this work for you?

from actual import Actual
from datetime import datetime
import decimal
from actual.queries import get_accounts, reconcile_transaction, get_ruleset

csv_data = [["Date", "Payee", "Notes", "Category", "Amount", "Cleared", "imported_ID"]]

URL_ACTUAL = "http://localhost:5006"
PASSWORD_ACTUAL = "mypass"
FILE_ACTUAL = "CSV Importer"

with Actual(base_url=URL_ACTUAL, password=PASSWORD_ACTUAL, file=FILE_ACTUAL) as actual:
    accounts = get_accounts(actual.session)
    ruleset = get_ruleset(actual.session)
    added_transactions = []
    for account in accounts:
        for row in csv_data:
            # here, we define the basic information from the file
            date, payee, notes, category, amount, cleared, imported_ID = (
                datetime.strptime(row[0], "%Y-%m-%d").date(),  # transform to date
                row[1],
                row[2],
                row[3],
                decimal.Decimal(row[4]),
                row[5] == "Cleared",
                row[6],
            )
            t = reconcile_transaction(
                actual.session,
                date,
                account,
                payee,
                notes,
                category,
                amount,
                imported_ID,
                cleared=cleared,
                imported_payee=payee,
                already_matched=added_transactions,
            )
            actual.session.flush()  # flush to load the ids and relationships
            ruleset.run(t)  # run the rule here
            added_transactions.append(t)
    actual.commit()

@tcpr1
Copy link
Author

tcpr1 commented Sep 8, 2024

It worked now! Appreciate your help!

@tcpr1
Copy link
Author

tcpr1 commented Sep 13, 2024

Caught a new bug, now it's when running ruleset.run(t). The code example is basically the same as you sent above.
The error happens when my budget has rules with "imported payee - any option - name" (similar to issue #67) or has transactions automatically added by schedules rules.

This raised due to scheduled transactions. Ir only happens then the date from a transaction "t" is equal a trasaction added by a schedule. When I delete the schedule, it stops the error:

AttributeError: 'NoneType' object has no attribute 'date'

Traceback:
File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/exec_code.py", line 88, in exec_func_with_error_handling
result = func()
^^^^^^
File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 590, in code_to_exec
exec(code, module.dict)
File "/app/streamlit_app.py", line 91, in
pluggy_sync(URL_ACTUAL, PASSWORD_ACTUAL, FILE_ACTUAL, start_date, end_date)
File "/app/functions.py", line 256, in pluggy_sync
data_to_actual(csv_data, actual.session, account)
File "/app/functions.py", line 163, in data_to_actual
ruleset.run(t) # run the rule here
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 555, in run
the order 'pre' -> None -> 'post'. You can provide a value to run only a certain stage of rules."""
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 544, in _run
for t in transaction:
File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 495, in run
def run(self, transaction: Transactions) -> bool:
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 490, in evaluate
def evaluate(self, transaction: Transactions) -> bool:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 490, in
def evaluate(self, transaction: Transactions) -> bool:
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 302, in run
attr = get_attribute_by_table_name(Transactions.tablename, self.field)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 185, in condition_evaluation
# https://github.com/actualbudget/actual/blob/98a7aac73667241da350169e55edd2fc16a6687f/packages/loot-core/src/server/accounts/rules.ts#L302-L304
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/actual/schedules.py", line 166, in is_approx
before = self.before(date)
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/actual/schedules.py", line 233, in before
return self.do_skip_weekend(dt_start, before_datetime).date()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This raised due to " imported payee - any option - name " rule. Deleting the rule, stops the error:

ValidationError: 1 validation error for list[function-after[check_operation_type(), function-after[convert_value(), Condition]]] 0.type Input should be 'date', 'id', 'string', 'number' or 'boolean' [type=enum, input_value='imported_payee', input_type=str] For further information visit https://errors.pydantic.dev/2.9/v/enum

Traceback:
File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/exec_code.py", line 88, in exec_func_with_error_handling
result = func()
^^^^^^
File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 590, in code_to_exec
exec(code, module.dict)
File "/app/streamlit_app.py", line 91, in
pluggy_sync(URL_ACTUAL, PASSWORD_ACTUAL, FILE_ACTUAL, start_date, end_date)
File "/app/functions.py", line 256, in pluggy_sync
data_to_actual(csv_data, actual.session, account)
File "/app/functions.py", line 136, in data_to_actual
ruleset = get_ruleset(actual_session)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/actual/queries.py", line 604, in get_ruleset
:param s: session from Actual local database.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pydantic/type_adapter.py", line 135, in wrapped
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pydantic/type_adapter.py", line 384, in validate_json
return self.validator.validate_json(data, strict=strict, context=context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

@bvanelli
Copy link
Owner

bvanelli commented Sep 14, 2024

Do your schedule happens to have an end date? Could you share how it is defined?

@tcpr1
Copy link
Author

tcpr1 commented Sep 14, 2024

Sure, here it is:

image

image

@bvanelli
Copy link
Owner

I generated a new release, could you update and check if it still fails?

@tcpr1
Copy link
Author

tcpr1 commented Sep 15, 2024

It failed 😔 However, there is improvement! It did not failed now because of the "imported payee" rule. It only fails because of a scheduled rule.

Oh, while typing here I was testing to find exactly what was causing the error and I found it! It's this date rule inside the schedule:
image

It was set to "before". Changing to "after" or unselecting it stops the error.

The error output:

> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
> Cell In[4], [line 15](vscode-notebook-cell:?execution_count=4&line=15)
>      [13](vscode-notebook-cell:?execution_count=4&line=13) start_date = "2024-09-08"
>      [14](vscode-notebook-cell:?execution_count=4&line=14) end_date = "2024-09-10"
> ---> [15](vscode-notebook-cell:?execution_count=4&line=15) pluggy_sync(URL_ACTUAL, PASSWORD_ACTUAL, FILE_ACTUAL, start_date, end_date)
> 
> File [z:\docker\code-server\config\actual-pluggy-py\functions.py:257](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:257), in pluggy_sync(URL_ACTUAL, PASSWORD_ACTUAL, FILE_ACTUAL, start_date, end_date)
>     [255](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:255)             if pluggy_status: # if pluggy connection failed, do nothing
>     [256](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:256)                 print(f"Starting Actual reconciliation for {accName}")
> --> [257](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:257)                 data_to_actual(csv_data, actual.session, account)
>     [259](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:259) actual.commit()
> 
> File [z:\docker\code-server\config\actual-pluggy-py\functions.py:164](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:164), in data_to_actual(csv_data, actual_session, account)
>     [162](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:162) actual_session.flush()  # flush to load the ids and relationships
>     [163](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:163) # print(t)
> --> [164](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:164) ruleset.run(t)  # run the rule here
>     [165](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:165) added_transactions.append(t)
>     [167](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:167) if t.changed():
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:594](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:594), in RuleSet.run(self, transaction, stage)
>     [592](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:592) if stage == "all":
>     [593](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:593)     self._run(transaction, "pre")
> --> [594](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:594)     self._run(transaction, None)
>     [595](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:595)     self._run(transaction, "post")
>     [596](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:596) else:
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:583](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:583), in RuleSet._run(self, transaction, stage)
>     [581](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:581)         rule.run(t)
>     [582](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:582) else:
> --> [583](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:583)     rule.run(transaction)
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:534](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:534), in Rule.run(self, transaction)
>     [531](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:531) def run(self, transaction: Transactions) -> bool:
>     [532](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:532)     """Runs the rule on the transaction, calling evaluate, and if the return is `True` then running each of
>     [533](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:533)     the actions."""
> --> [534](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:534)     if condition_met := self.evaluate(transaction):
>     [535](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:535)         splits = self.set_split_amount(transaction)
>     [536](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:536)         if splits:
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:529](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:529), in Rule.evaluate(self, transaction)
>     [527](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:527) """Evaluates the rule on the transaction, without applying any action."""
>     [528](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:528) op = any if self.operation == "or" else all
> --> [529](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:529) return op(c.run(transaction) for c in self.conditions)
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:529](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:529), in <genexpr>(.0)
>     [527](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:527) """Evaluates the rule on the transaction, without applying any action."""
>     [528](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:528) op = any if self.operation == "or" else all
> --> [529](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:529) return op(c.run(transaction) for c in self.conditions)
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:322](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:322), in Condition.run(self, transaction)
>     [320](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:320) true_value = get_value(getattr(transaction, attr), self.type)
>     [321](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:321) self_value = self.get_value()
> --> [322](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:322) return condition_evaluation(self.op, true_value, self_value, self.options)
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:200](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:200), in condition_evaluation(op, true_value, self_value, options)
>     [198](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:198)     interval = datetime.timedelta(days=2)
>     [199](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:199)     if isinstance(self_value, Schedule):
> --> [200](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:200)         return self_value.is_approx(true_value, interval)
>     [201](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:201) else:
>     [202](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:202)     # Actual uses 7.5% of the value as threshold
>     [203](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:203)     # https://github.com/actualbudget/actual/blob/243703b2f70532ec1acbd3088dda879b5d07a5b3/packages/loot-core/src/shared/rules.ts#L261-L263
>     [204](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:204)     interval = round(abs(self_value) * 0.075, 2)
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\schedules.py:166](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:166), in Schedule.is_approx(self, date, interval)
>     [164](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:164) if date < self.start or (self.end_mode == EndMode.ON_DATE and self.end_date < date):
>     [165](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:165)     return False
> --> [166](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:166) before = self.before(date)
>     [167](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:167) after = self.xafter(date, 1)
>     [168](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:168) if before and (before - interval <= date <= before + interval):
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\schedules.py:233](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:233), in Schedule.before(self, date)
>     [231](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:231) if not before_datetime:
>     [232](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:232)     return None
> --> [233](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:233) return self.do_skip_weekend(dt_start, before_datetime).date()

@bvanelli
Copy link
Owner

Hello @tcpr1, thanks for your last message, I could finally reproduce the reason why it was crashing. Turns out I was comparing the wrong cutoff date when resolving the before weekend mode.

It should be fixed in the soon to be released version 0.5.1.

@tcpr1
Copy link
Author

tcpr1 commented Sep 23, 2024

No more errors!

@bvanelli
Copy link
Owner

Hey @tcpr1 , I'll then close this issue with #85 after including more documentation and one example using your use case as a base.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants