Mailer not working for data-importer?

[psychowood]

Hi all,
I'm opening a discussion here since I'm not really sure that this is not a problem on my side rather than a bug, but I've been struggling with this for a few hours and I can't seem to find a solution, so here I am.

Context:

• firefly-iii and data-importer are configured via docker-compose, working as expected besides this issue
• both apps are in the same compose file, in the same network, behind the same reverse proxy (I also tried by exposing data-importer directly from the docker host, same result)
• firefly-iii mail are working correctly, I can send and receive the test email
• data-importer has exactly the same configuration of firefly-iii (w/ different names for env variables where needed of course) in regard of mail host, user, password, from account and destination account (I set the same email of the user in firefly-iii).

Thing is, I can't seem to trigger the report. Even stranger, there's no trace of anything mail related in the logs (I enabled all the debug modes I could find), not even the message 'Now in sendReportOverMail' I'd expect from the source code.

exim4 is not reporting any connection from data-importer, while it does for firefly-iii.

I tried both the 1.4.0 version and the 1.3.12 of data-importer, same result. Changing the mailer to 'log' does not change the behaviour.

Am I missing something?

Thanks, and happy new year :)

---

For reference, these are the relevant variables reported from the running data-importer container by portainer:

ENABLE_MAIL_REPORT 	true
LOG_LEVEL 	debug
LOG_RETURN_JSON 	true
MAIL_DESTINATION 	xxxxx@mydomain
MAIL_ENCRYPTION 	null
MAIL_FROM_ADDRESS 	yyyyy@mydomain
MAIL_HOST 	exim4
MAIL_MAILER 	smtp
MAIL_PASSWORD 	null
MAIL_PORT 	8025
MAIL_USERNAME 	null

while these are the ones for firefly-iii:

MAIL_ENCRYPTION 	null
MAIL_FROM 	yyyy@mydomain
MAIL_HOST 	exim4
MAIL_MAILER 	smtp
MAIL_PASSWORD 	null
MAIL_PORT 	8025
MAIL_SENDMAIL_COMMAND 	
MAIL_USERNAME 	null

[JC5]

How do you run the import routine? Over the UI, or using the command line?

[psychowood]

Since there is no report sent, would you consider adding something that allows to save the output messages in a local file? Something in plain html/javascript, no server side generation.

I can prepare a jquery script if it can be useful, it should have minimal impacts on the current page.

[JC5]

I added the report back to the UI routine, it should email now (in develop).

[psychowood]

Tested develop branch a few moments ago:

• email was sent (yay! 💯 )
• report has some issues, it looks like all three arrays (errors, warning, messages) are populated with the 9 messages I had discarding the 217 errors (lots of duplicated transactions)

Logs:

fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: Status: submission_running
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: Messages: 9, warnings: 0, errors: 217
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: Now in App\Services\Shared\Import\Status\SubmissionStatusManager::storeSubmissionStatus(submission-ikoUfurfyRWH): submission_done
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: Messages: 9, warnings: 0, errors: 217
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: Created event ImportedTransactions with filtering (2)
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: Array contains 9 message(s)
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: Array contains 9 warning(s)
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: Array contains 9 error(s)
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: Now in sendReportOverMail
fireflyiiiimporter    | [2024-01-04 18:46:16] local.INFO: Will send report message.
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: Messages count: 9
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: Warnings count: 9
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: Errors count  : 9
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: If no error below this line, mail was sent!
fireflyiiiimporter    | [2024-01-04 18:46:16] local.DEBUG: If no error above this line, mail was sent!
fireflyiiiimporter    | 172.23.0.28 - - [04/Jan/2024:18:45:33 +0100] "POST /import/submit/start?identifier=submission-ikoUfurfyRWH HTTP/1.1" 200 1150 "https://fireflyiiiimporter.lan/import/submit" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0) Gecko/20100101 Firefox/121.0"
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: isReadyForFileStep("submit")
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: isReadyForFileStep: Return true
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: Now at App\Http\Controllers\Import\SubmitController::status(submission-ikoUfurfyRWH)
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: Now in startOrFindJob(submission-ikoUfurfyRWH)
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: Try to see if file exists for job submission-ikoUfurfyRWH.
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: Status file exists for job submission-ikoUfurfyRWH.
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: Status: submission_done
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: Messages: 9, warnings: 0, errors: 217
fireflyiiiimporter    | 172.23.0.28 - - [04/Jan/2024:18:46:17 +0100] "GET /import/submit/status?identifier=submission-ikoUfurfyRWH HTTP/1.1" 200 27252 "https://fireflyiiiimporter.lan/import/submit" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0) Gecko/20100101 Firefox/121.0"
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: isReadyForFileStep("submit")
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: isReadyForFileStep: Return true
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: Now at App\Http\Controllers\Import\SubmitController::status(submission-ikoUfurfyRWH)
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: Now in startOrFindJob(submission-ikoUfurfyRWH)
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: Try to see if file exists for job submission-ikoUfurfyRWH.
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: Status file exists for job submission-ikoUfurfyRWH.
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: Status: submission_done
fireflyiiiimporter    | [2024-01-04 18:46:17] local.DEBUG: Messages: 9, warnings: 0, errors: 217
fireflyiiiimporter    | 172.23.0.28 - - [04/Jan/2024:18:46:17 +0100] "GET /import/submit/status?identifier=submission-ikoUfurfyRWH HTTP/1.1" 200 27252 "https://fireflyiiiimporter.lan/import/submit" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0) Gecko/20100101 Firefox/121.0"

Partial (for privacy reasons) screen of the email

[JC5]

Nice, I fixed that three hours ago 😁 , it should be part of the next Docker build tonight.

[psychowood]

We need instant CI builds :D (just joking ofc)
Thanks for the efforts.