Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Maven package ecosystem not checking dependencies from my private repo #5523

Closed
gianielsevier opened this issue Aug 12, 2022 · 8 comments
Closed

Maven package ecosystem not checking dependencies from my private repo #5523

gianielsevier opened this issue Aug 12, 2022 · 8 comments
Labels
L: java:maven Maven packages via Maven T: bug 🐞 Something isn't working

Comments

@gianielsevier
Copy link

gianielsevier commented Aug 12, 2022 •
edited
Loading

Package ecosystem
maven
Package manager version
3.8.6
Language version
java11, java17
Manifest location and content before the Dependabot update
pom.xml

If using GitHub-native Dependabot, attach your dependabot.yml file or provide a link to it

version: 2
registries:
  my-artifactory:
    type: maven-repository
    url: https://my.internal.repo
    username: MY_USER
    password: ${{secrets.MY_SECRET}}
updates:
  - package-ecosystem: "maven"
    directory: "/"
    registries:
      - my-artifactory
    schedule:
      interval: "daily"

What you expected to see, versus what you actually saw
I was expecting dependabot check dependencies from my private maven repo. But it is going to repo.maven.apache
Native package manager behavior

proxy | time="2022-08-12T09:55:35Z" level=info msg="proxy starting" commit=d1feaa99254076b1f2be38bedebf2e3fe49ef1c6
  proxy | 2022/08/12 09:55:35 Listening (:1080)
updater | 2022-08-12T09:55:35.249798454 [anonymous-instance:main:WARN:src/firecracker/src/main.rs:370] You are using a deprecated parameter: --seccomp-level 2, that will be removed in a future version.
updater | 2022-08-12T09:55:35.273297317 [439162729:main:WARN:src/devices/src/legacy/serial.rs:432] Detached the serial input due to peer close/error.
updater | time="2022-08-12T09:55:36Z" level=info msg="guest starting" commit=03c82863413a174048b5d5d081fe33c52016da94
updater | time="2022-08-12T09:55:36Z" level=info msg="starting job..." fetcher_timeout=5m0s job_id=439162729 updater_timeout=45m0s updater_version=0.207.0-8bdfd36a8d3a47a9944976c94af2e0105382e08f
updater | I, [2022-08-12T09:55:38.142880 #7]  INFO -- sentry: ** [Raven] Raven 3.1.2 ready to catch errors
updater | To use retry middleware with Faraday v2.0+, install `faraday-retry` gem
updater | INFO <job_439162729> Starting job processing
  proxy | 2022/08/12 09:55:40 [002] GET https://api.github.com:443/repos/internal-github-repo/my-app
  proxy | 2022/08/12 09:55:40 [002] * authenticating github api request
  proxy | 2022/08/12 09:55:40 [002] 200 https://api.github.com:443/repos/internal-github-repo/my-app
  proxy | 2022/08/12 09:55:40 [004] GET https://api.github.com:443/repos/internal-github-repo/my-app/git/refs/heads/main
  proxy | 2022/08/12 09:55:40 [004] * authenticating github api request
  proxy | 2022/08/12 09:55:40 [004] 200 https://api.github.com:443/repos/internal-github-repo/my-app/git/refs/heads/main
  proxy | 2022/08/12 09:55:40 [006] GET https://api.github.com:443/repos/internal-github-repo/my-app/contents/pom.xml?ref=7e1aa9a79a954f4f828898c8997c422d789ea88b
  proxy | 2022/08/12 09:55:40 [006] * authenticating github api request
  proxy | 2022/08/12 09:55:40 [006] 200 https://api.github.com:443/repos/internal-github-repo/my-app/contents/pom.xml?ref=7e1aa9a79a954f4f828898c8997c422d789ea88b
  proxy | 2022/08/12 09:55:40 [008] GET https://api.github.com:443/repos/internal-github-repo/my-app/contents/.mvn?ref=7e1aa9a79a954f4f828898c8997c422d789ea88b
  proxy | 2022/08/12 09:55:40 [008] * authenticating github api request
  proxy | 2022/08/12 09:55:41 [008] 404 https://api.github.com:443/repos/internal-github-repo/my-app/contents/.mvn?ref=7e1aa9a79a954f4f828898c8997c422d789ea88b
updater | INFO <job_439162729> Finished job processing
updater | time="2022-08-12T09:55:41Z" level=info msg="task complete" container_id=job-439162729-file-fetcher exit_code=0 job_id=439162729 step=fetcher
updater | I, [2022-08-12T09:55:41.990806 #7]  INFO -- sentry: ** [Raven] Raven 3.1.2 ready to catch errors
updater | To use retry middleware with Faraday v2.0+, install `faraday-retry` gem
updater | INFO <job_439162729> Starting job processing
updater | INFO <job_439162729> Starting update job for internal-github-repo/my-app
updater | INFO <job_439162729> Checking if org.mapstruct:mapstruct  needs updating
  proxy | 2022/08/12 09:55:44 [012] GET https://my.internal.repo:443/artifactory/maven-ssdr-libs-releases-virtual/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:55:44 [012] * authenticating maven repository request (host: my.internal.repo)
  proxy | 2022/08/12 09:56:04 [014] GET https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:04 [014] 404 https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:04 [016] GET https://repo.maven.apache.org:443/maven2/org/mapstruct/mapstruct/maven-metadata.xml
  proxy | 2022/08/12 09:56:04 [016] 200 https://repo.maven.apache.org:443/maven2/org/mapstruct/mapstruct/maven-metadata.xml
  proxy | 2022/08/12 09:56:04 [018] HEAD https://repo.maven.apache.org:443/maven2/org/mapstruct/mapstruct/1.5.2.Final/mapstruct-1.5.2.Final.jar
  proxy | 2022/08/12 09:56:04 [018] 200 https://repo.maven.apache.org:443/maven2/org/mapstruct/mapstruct/1.5.2.Final/mapstruct-1.5.2.Final.jar
updater | INFO <job_439162729> Latest version is 1.5.2.Final
updater | INFO <job_439162729> No update needed for org.mapstruct:mapstruct 
updater | INFO <job_439162729> Checking if org.springframework.security:spring-security-test  needs updating
  proxy | 2022/08/12 09:56:04 [020] GET https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:04 [020] 404 https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:04 [022] GET https://repo.maven.apache.org:443/maven2/org/springframework/security/spring-security-test/maven-metadata.xml
  proxy | 2022/08/12 09:56:04 [022] 200 https://repo.maven.apache.org:443/maven2/org/springframework/security/spring-security-test/maven-metadata.xml
  proxy | 2022/08/12 09:56:04 [024] HEAD https://repo.maven.apache.org:443/maven2/org/springframework/security/spring-security-test/5.7.2/spring-security-test-5.7.2.jar
  proxy | 2022/08/12 09:56:04 [024] 200 https://repo.maven.apache.org:443/maven2/org/springframework/security/spring-security-test/5.7.2/spring-security-test-5.7.2.jar
updater | INFO <job_439162729> Latest version is 5.7.2
updater | INFO <job_439162729> No update needed for org.springframework.security:spring-security-test 
updater | INFO <job_439162729> Checking if my.internal.groupid:internal-dependency-one  needs updating
  proxy | 2022/08/12 09:56:04 [026] GET https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:04 [026] 404 https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:04 [028] GET https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-dependency-one/maven-metadata.xml
  proxy | 2022/08/12 09:56:04 [028] 404 https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-dependency-one/maven-metadata.xml
updater | INFO <job_439162729> Latest version is 
updater | INFO <job_439162729> No update needed for my.internal.groupid:internal-dependency-one 
updater | INFO <job_439162729> Checking if org.springframework.boot:spring-boot-maven-plugin  needs updating
  proxy | 2022/08/12 09:56:04 [030] GET https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:04 [030] 404 https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:04 [032] GET https://repo.maven.apache.org:443/maven2/org/springframework/boot/spring-boot-maven-plugin/maven-metadata.xml
  proxy | 2022/08/12 09:56:04 [032] 200 https://repo.maven.apache.org:443/maven2/org/springframework/boot/spring-boot-maven-plugin/maven-metadata.xml
  proxy | 2022/08/12 09:56:04 [034] HEAD https://repo.maven.apache.org:443/maven2/org/springframework/boot/spring-boot-maven-plugin/2.7.2/spring-boot-maven-plugin-2.7.2.jar
  proxy | 2022/08/12 09:56:04 [034] 200 https://repo.maven.apache.org:443/maven2/org/springframework/boot/spring-boot-maven-plugin/2.7.2/spring-boot-maven-plugin-2.7.2.jar
updater | INFO <job_439162729> Latest version is 2.7.2
updater | INFO <job_439162729> No update needed for org.springframework.boot:spring-boot-maven-plugin 
updater | INFO <job_439162729> Checking if my.internal.groupid:internal-dependency-two 1.5.1-SNAPSHOT needs updating
  proxy | 2022/08/12 09:56:04 [036] GET https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:04 [036] 404 https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:05 [038] GET https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-dependency-two/maven-metadata.xml
  proxy | 2022/08/12 09:56:05 [038] 404 https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-dependency-two/maven-metadata.xml
updater | INFO <job_439162729> Latest version is 
updater | INFO <job_439162729> Requirements to unlock update_not_possible
updater | INFO <job_439162729> Requirements update strategy 
updater | INFO <job_439162729> No update possible for my.internal.groupid:internal-dependency-two 1.5.1
updater | INFO <job_439162729> Checking if my.internal.groupid:internal-dependency-three  needs updating
  proxy | 2022/08/12 09:56:05 [040] GET https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:05 [040] 404 https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:05 [042] GET https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-dependency-three/maven-metadata.xml
  proxy | 2022/08/12 09:56:05 [042] 404 https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-dependency-three/maven-metadata.xml
updater | INFO <job_439162729> Latest version is 
updater | INFO <job_439162729> No update needed for my.internal.groupid:internal-dependency-three 
updater | INFO <job_439162729> Checking if my.internal.groupid:internal-parent 1.5.0 needs updating
  proxy | 2022/08/12 09:56:05 [044] GET https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:05 [044] 404 https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:56:05 [046] GET https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/maven-metadata.xml
  proxy | 2022/08/12 09:56:05 [046] 404 https://repo.maven.apache.org:443/maven2/my/internal/groupid/internal-parent/maven-metadata.xml
updater | INFO <job_439162729> Latest version is 
updater | INFO <job_439162729> Requirements to unlock update_not_possible
updater | INFO <job_439162729> Requirements update strategy 
updater | INFO <job_439162729> No update possible for my.internal.groupid:internal-parent 1.5.0
updater | INFO <job_439162729> Finished job processing
updater | time="2022-08-12T09:56:05Z" level=info msg="task complete" container_id=job-439162729-updater exit_code=0 job_id=439162729 step=updater

🕹 Bonus points: Smallest manifest that reproduces the issue

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>my.internal.groupId</groupId>
        <artifactId>internal-parent</artifactId>
        <version>1.5.0</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <artifactId>my-app</artifactId>
    <version>1.5.0</version>
    <packaging>jar</packaging>

    <name>my-app-name</name>
    <description>My Description</description>

    <dependencies>
        <!--	internal dependencies	-->
        <dependency>
            <groupId>internal.groupId</groupId>
            <artifactId>internal-dependency-one</artifactId>
        </dependency>
       <dependency>
            <groupId>internal.groupId</groupId>
            <artifactId>internal-dependency-two</artifactId>
        </dependency>
        <dependency>
            <groupId>org.mapstruct</groupId>
            <artifactId>mapstruct</artifactId>
        </dependency>
        <!--	test	-->
        <dependency>
            <groupId>org.springframework.security</groupId>
            <artifactId>spring-security-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>
@gianielsevier gianielsevier added the T: bug 🐞 Something isn't working label Aug 12, 2022
@gianielsevier
Copy link
Author

I trust the dependabot is not able to connect to my internal maven repo but I can't see any authentication error or timeout

@jeffwidman
Copy link
Member

Is this a duplicate of #2291?

@pavera
Copy link
Contributor

pavera commented Aug 17, 2022

I believe you need a <registries> section in your pom.xml to tell maven to scan the internal repo. Dependabot parses the pom.xml and uses this section defined in your pom.xml to decide which registries to scan.

@gianielsevier
Copy link
Author

Hi, @pavera thanks for your reply. Do you mean ?
If yes, it is already coming from my parent pom.xml. As we have many applications we keep this config centralized in our framework:

<parent>
        <groupId>my.internal.groupId</groupId>
        <artifactId>internal-parent</artifactId>
        <version>1.5.0</version>
        <relativePath/> <!-- lookup parent from repository -->
</parent>

Kind regards,
Giani Segatto

@pavera pavera added the L: java:maven Maven packages via Maven label Aug 25, 2022
@jakecoffman jakecoffman mentioned this issue Oct 13, 2022
Merged
@jakecoffman
Copy link
Member

@gianielsevier I just deployed #5884 if you would like to manually trigger a run to see if it is fixed now.

@jakecoffman
Copy link
Member

I trust the dependabot is not able to connect to my internal maven repo but I can't see any authentication error or timeout

Ah, I see what you are saying now. This call is timing out:

updater | INFO <job_439162729> Checking if org.mapstruct:mapstruct  needs updating
  proxy | 2022/08/12 09:55:44 [012] GET https://my.internal.repo:443/artifactory/maven-ssdr-libs-releases-virtual/my/internal/groupid/internal-parent/1.5.0/internal-parent-1.5.0.pom
  proxy | 2022/08/12 09:55:44 [012] * authenticating maven repository request (host: my.internal.repo)

But we don't see the stack because the job ends too quickly. I've been testing timeout issues internally and the stack often comes several minutes after the initial call.

@gianielsevier
Copy link
Author

Hi, @jakecoffman thanks for working on this issue. I'll try to run it again

@jeffwidman
Copy link
Member

@gianielsevier any update?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
L: java:maven Maven packages via Maven T: bug 🐞 Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jeffwidman @pavera @jakecoffman @gianielsevier