Problem with Firefly III Data Import

[Joca9227]

Hello everyone,
I'm new on Firefly III. I could install the app but i doesn't understand how to proper install Firefly III Data Import.
I'm using portainer.
Firefly version 6.1.19
I use that as variables:

APP_DEBUG=false
[email protected]
APP_KEY=*****
DEFAULT_LANGUAGE=en_US
DEFAULT_LOCALE=equal
TZ=Europe/Amsterdam
TRUSTED_PROXIES=
LOG_CHANNEL=stack
APP_LOG_LEVEL=notice
AUDIT_LOG_LEVEL=emergency
AUDIT_LOG_CHANNEL=
PAPERTRAIL_HOST=
PAPERTRAIL_PORT=
DB_CONNECTION=mysql
DB_HOST=db
DB_PORT=3306
DB_DATABASE=firefly
DB_USERNAME=firefly
DB_PASSWORD=*****
DB_SOCKET=
MYSQL_USE_SSL=false
MYSQL_SSL_VERIFY_SERVER_CERT=true
MYSQL_SSL_CAPATH=/etc/ssl/certs/
MYSQL_SSL_CA=
MYSQL_SSL_CERT=
MYSQL_SSL_KEY=
MYSQL_SSL_CIPHER=
PGSQL_SSL_MODE=prefer
PGSQL_SSL_ROOT_CERT=null
PGSQL_SSL_CERT=null
PGSQL_SSL_KEY=null
PGSQL_SSL_CRL_FILE=null
PGSQL_SCHEMA=public
CACHE_DRIVER=file
SESSION_DRIVER=file
REDIS_SCHEME=tcp
REDIS_PATH=
REDIS_HOST=127.0.0.1
REDIS_PORT=6379
REDIS_USERNAME=
REDIS_PASSWORD=
REDIS_DB="0"
REDIS_CACHE_DB="1"
COOKIE_PATH="/"
COOKIE_DOMAIN=
COOKIE_SECURE=false
COOKIE_SAMESITE=lax
MAIL_MAILER=log
MAIL_HOST=null
MAIL_PORT=2525
[email protected]
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null
MAIL_SENDMAIL_COMMAND=
MAILGUN_DOMAIN=
MAILGUN_SECRET=
MAILGUN_ENDPOINT=api.mailgun.net
MANDRILL_SECRET=
SPARKPOST_SECRET=
SEND_ERROR_MESSAGE=true
SEND_REPORT_JOURNALS=true
ENABLE_EXTERNAL_MAP=false
ENABLE_EXCHANGE_RATES=false
ENABLE_EXTERNAL_RATES=false
MAP_DEFAULT_LAT=51.983333
MAP_DEFAULT_LONG=5.916667
MAP_DEFAULT_ZOOM=6
VALID_URL_PROTOCOLS=
AUTHENTICATION_GUARD=web
AUTHENTICATION_GUARD_HEADER=REMOTE_USER
AUTHENTICATION_GUARD_EMAIL=
PASSPORT_PRIVATE_KEY=
PASSPORT_PUBLIC_KEY=
CUSTOM_LOGOUT_URL=
DISABLE_FRAME_HEADER=false
DISABLE_CSP_HEADER=false
TRACKER_SITE_ID=
TRACKER_URL=
ALLOW_WEBHOOKS=false
STATIC_CRON_TOKEN=*****
DKR_BUILD_LOCALE=false
DKR_CHECK_SQLITE=true
APP_NAME=FireflyIII
BROADCAST_DRIVER=log
QUEUE_DRIVER=sync
CACHE_PREFIX=firefly
PUSHER_KEY=
IPINFO_TOKEN=
PUSHER_SECRET=
PUSHER_ID=
DEMO_USERNAME=
DEMO_PASSWORD=
FIREFLY_III_LAYOUT=v1
APP_URL=http://localhost
FIREFLY_III_URL=http://firefly.jffp.home:8080
VANITY_URL=http://firefly.jffp.home
FIREFLY_III_ACCESS_TOKEN=
FIREFLY_III_CLIENT_ID=
NORDIGEN_ID=
NORDIGEN_KEY=
NORDIGEN_SANDBOX=false
SPECTRE_APP_ID=
SPECTRE_SECRET=
USE_CACHE=false
IGNORE_DUPLICATE_ERRORS=false
IGNORE_NOT_FOUND_TRANSACTIONS=false
AUTO_IMPORT_SECRET=
CAN_POST_AUTOIMPORT=false
CAN_POST_FILES=false
IMPORT_DIR_ALLOWLIST=
FALLBACK_IN_DIR=false
VERIFY_TLS_SECURITY=true
JSON_CONFIGURATION_DIR=
CONNECTION_TIMEOUT=31.41
LOG_RETURN_JSON=false
LOG_LEVEL=debug
ENABLE_MAIL_REPORT=false
EXPECT_SECURE_URL=false
[email protected]
[email protected]
POSTMARK_TOKEN=
QUEUE_CONNECTION=sync
SESSION_LIFETIME=120
IS_EXTERNAL=false
ASSET_URL=
MYSQL_RANDOM_ROOT_PASSWORD=yes
MYSQL_USER=firefly
MYSQL_PASSWORD=*****
MYSQL_DATABASE=firefly

I create a new Oauth client
and when i use on Redirect URL http://ip:81/callback i got an error 500
and if i use anything else i get error : "invalid_client"
I can't connect me to FIREFLY_III_URL http://firefly.jffp.home:8080
I can use the app with any problem but not the Import data
Could someone help me please

[JC5]

Did you uncheck the box "Confidential" when you created the client?

[Joca9227]

Yes i did

[JC5]

Sorry, then I don't know what it could be. "Invalid client" usually points to copy/paste errors or indeed a forgotten checkbox.

[Coder1304]

@Joca9227 I have same problem, this is some kind of docker image/container issue. The DNS for network is resolving IP correctly.
But importer can't get access to port.

When I run curl from core container it work fine (I didn't provide token so I got 401)

root@app:/var/www/html# curl -vvv -X GET --connect-timeout 10 'http://firefly_iii_core:8080/api/v1/about?'
Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying 172.22.0.2:8080...
* Connected to firefly_iii_core (172.22.0.2) port 8080 (#0)
> GET /api/v1/about? HTTP/1.1
> Host: firefly_iii_core:8080
> User-Agent: curl/7.74.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 Unauthorized
< Date: Sat, 03 Aug 2024 22:17:39 GMT
< Server: Apache
< Cache-Control: no-cache, private
< X-Frame-Options: deny
< Content-Security-Policy: default-src 'none'; object-src 'none'; script-src 'unsafe-eval' 'strict-dynamic' 'nonce-TXOR9+WVlmXL41WrWRgtKg=='; style-src 'unsafe-inline' 'self'; base-uri 'self'; font-src 'self' data:; connect-src 'self' ; img-src 'self' data: 'nonce-TXOR9+WVlmXL41WrWRgtKg==' ; manifest-src 'self'; form-action 'self'
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Referrer-Policy: no-referrer
< X-Permitted-Cross-Domain-Policies: none
< X-Robots-Tag: none
< Feature-Policy: geolocation 'none'; midi 'none'; sync-xhr 'self'; microphone 'none'; camera 'none'; magnetometer 'none'; gyroscope 'none'; fullscreen 'self'; payment 'none'
< Transfer-Encoding: chunked
< Content-Type: application/json
< 
* Connection #0 to host firefly_iii_core left intact
{"message":"Unauthenticated","exception":"AuthenticationException"}

also no issues when connecting from outside using mapped port, server is responding with 401:

$ curl -X GET 'http://192.168.0.250:48080/api/v1/about?'
{"message":"Unauthenticated","exception":"AuthenticationException"}

When I run the same from importer container, I am getting timeout:

root@importer:/var/www/html# curl -vvv -X GET --connect-timeout 10 'http://firefly_iii_core:8080/api/v1/about?'
Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying 172.22.0.2:8080...
* Connection timed out after 10001 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 10001 milliseconds
root@importer:/var/www/html# curl -X GET 'http://192.168.0.250:48080/api/v1/about?'
^C
root@importer:/var/www/html# curl -X GET -vvv --connect-timeout 10 'http://192.168.0.250:48080/api/v1/about?'
Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying 192.168.0.250:48080...
* Connection timed out after 10000 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 10000 milliseconds
root@importer:/var/www/html# curl -X GET -vvv --connect-timeout 10 'http://172.17.0.1:48080/api/v1/about?'
Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying 172.17.0.1:48080...
* Connection timed out after 10000 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 10000 milliseconds

@JC5 I think it is due to missing "EXPOSE 8080" in core Dockerfile.

[Coder1304]

More from ChatGPT, take a look on point 2

Key Points about EXPOSE:

   1. Documentation: The EXPOSE directive documents the ports that the containerized application will use. This is helpful for anyone who needs to understand how to interact with the container.

   2. Internal Networking: While EXPOSE does not publish the port to the host, it does make the port available for inter-container communication within the same Docker network.

   3. Runtime Behavior: The EXPOSE directive does not actually open the port. To make the port accessible from the host, you need to use the -p or --publish option when running the container.

[JC5]

I'm not sure what the problem could be, but I can assure you EXPOSE 8080 is in the Dockerfile.

[Coder1304]

Yes, I see it now. I need to look further. I have checked "docker image history" and indeed it is there so I am using correct image.

fireflyiii/core            latest    aec4bdf05bff   13 days ago    637MB

<missing>      4 weeks ago   EXPOSE map[8080/tcp:{}]                         0B        buildkit.dockerfile.v0

I try even scan the app container:

$docker run -it --network firefly_ext securecodebox/nmap nmap -Pn -p- app
Starting Nmap 7.80 ( https://nmap.org ) at 2024-08-04 20:50 UTC
Nmap scan report for app (172.22.0.2)
Host is up (0.00023s latency).
rDNS record for 172.22.0.2: firefly_iii_core.firefly_ext
Not shown: 65531 filtered ports
PORT      STATE  SERVICE
22/tcp    closed ssh
2222/tcp  closed EtherNetIP-1
3306/tcp  closed mysql
56562/tcp closed unknown

Nmap done: 1 IP address (1 host up) scanned in 104.61 seconds

What else could I check?

[s3w47m88]

I am experiencing the same issue. I noticed that the Authorize button leads to "localhost" not "localhost:port".

In my case Firefly Core is available at http://localhost:8495 and Firefly Importer is at http://localhost:8496 but the Import gives a 500 error because the URL is not including the port number:

http://localhost/oauth/authorize?client_id=5&redirect_uri=http%3A%2F%2Flocalhost%3A8496%2Fcallback&response_type=code&scope=&state={redacted}&code_challenge={redacted}&code_challenge_method=S256

So I changed it to my port for Importer (8496) to http://localhost:**8496**/oauth/authorize?client_id=5&redirect_uri=http%3A%2F%2Flocalhost%3A8496%2Fcallback&response_type=code&scope=&state={redacted}&code_challenge={redacted}&code_challenge_method=S256 and get the "authorize" screen.

Upon selecting Authorize button, however, it gives a 500 error.

Turns out this was due to: This was due to https://github.com/orgs/firefly-iii/discussions/6069#discussioncomment-9042219

But I still land on a 404: http://localhost:8496/oauth/authorize?client_id=6&redirect_uri=http%3A%2F%2Flocalhost%3A8496%2Fcallback&response_type=code&scope=&state=fmoGGv74Y9RuVFz8IUWmPFceqp4ixMH5LvLGE5Y4&code_challenge=IeaDU5K8bstEN4qWBJkynltPWEKLtA9ZcDw4eMGr2CI&code_challenge_method=S256

[JC5]

The URL seems to be correct, but shouldn't that port be 8495 according to your settings?

[s3w47m88]

I'm not quiet sure to be honest. I entered the URL in Firefly's settings provided by the Importer, which was 8496 - which is the port for the Importer. I think I tried port 8495 as well and still ended up at a 500 or 404. Sent with [Proton Mail](https://proton.me/) secure email.

…

On Monday, August 26th, 2024 at 11:06 PM, James Cole ***@***.***> wrote: The URL seems to be correct, but shouldn't that port be 8495 according to your settings? — Reply to this email directly, [view it on GitHub](#9097 (reply in thread)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AAFALCEOTOMYSM3ZNYDQO63ZTQJNBAVCNFSM6AAAAABLT2KAY2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANBVHEYTCNI). You are receiving this because you commented.Message ID: ***@***.***>