Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query output with deps order is wrong for proto output format #17087

Closed
xzxzzxzxzca opened this issue Dec 29, 2022 · 3 comments
Closed

Query output with deps order is wrong for proto output format #17087

xzxzzxzxzca opened this issue Dec 29, 2022 · 3 comments
Assignees
Labels
P2 We'll consider working on this in future. (Assignee optional) team-Performance Issues for Performance teams type: bug

Comments

@xzxzzxzxzca
Copy link

xzxzzxzxzca commented Dec 29, 2022

Description of the bug:

Bazel's protobuf output is not ordered the same way as the one in label format, even if --order_output=deps is set explicitly in both cases. "Label" output goes "top-down", listing dependencies last. So for example one could iterate the output in reverse order and see the source files first before seeing the rules consuming them. In proto output this is not the case, and source files seem to be output in no particular order (yet, consistent from run to run). Rules still seem to be ordered by dependencies though.
This is critical for any use-case where the output is supposed to be consumed and processed by another program, as building the workspace model from such output essentially can't rely on the ordering and would have to perform additional sorting on its own.

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Clone the examples repo
Go to e.g. cpp-tutorial/stage3 subdirectory
Query workspace first with outputs as labels: bazel query '//...:*' --order_output=deps --output label
Observe the output order:

//main:hello-world.stripped
//main:BUILD
//main:hello-world.dwp
//main:hello-world
//main:hello-world.cc
//main:hello-greet
//main:hello-greet.cc
//lib:BUILD
//:BUILD
//lib:hello-time
//lib:hello-time.h
//lib:hello-time.cc
//main:hello-greet.h

Now do the same query with protobuf output: bazel query '//...:*' --order_output=deps --output proto and compare the order of items to the previous output.
Although, proto is not very human-readable, it's still visible that the ordering is different. What's critical is that sources and generated files are absolutely out of place.

Which operating system are you running Bazel on?

MacOS

What is the output of bazel info release?

release 5.3.2-homebrew

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

https://github.com/bazelbuild/examples
f53d6699dc0fdfbf288b92345980ea8708e2db0a
f53d6699dc0fdfbf288b92345980ea8708e2db0a

Have you found anything relevant by searching the web?

Here's another somewhat relevant bug report I found: #12320

Any other information, logs, or outputs that you want to share?

No response

@sgowroji sgowroji added team-Rules-CPP Issues for C++ rules type: bug untriaged team-Configurability platforms, toolchains, cquery, select(), config transitions team-Performance Issues for Performance teams and removed team-Configurability platforms, toolchains, cquery, select(), config transitions team-Rules-CPP Issues for C++ rules labels Dec 29, 2022
@coeuvre coeuvre added P2 We'll consider working on this in future. (Assignee optional) and removed untriaged labels Jan 3, 2023
hvadehra pushed a commit that referenced this issue Feb 14, 2023
… topological order.

Fixes #17087.

RELNOTES[INC]: query --output=proto --order_output=deps now returns targets in topological order (previously there was no ordering).

PiperOrigin-RevId: 501548057
Change-Id: Ied86694fc6c7f9fc2243af1ce43bc778509cbe44
@xzxzzxzxzca
Copy link
Author

It looks like this is only fixed in Bazel 7. Will this be ported into 6?

@fmeum
Copy link
Collaborator

fmeum commented Jul 8, 2023

@bazel-io flag

@bazel-io bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Jul 8, 2023
@keertk
Copy link
Member

keertk commented Jul 10, 2023

@bazel-io fork 6.3.0

@bazel-io bazel-io removed the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Jul 10, 2023
keertk added a commit that referenced this issue Jul 10, 2023
… topological order. (#18870)

Fixes #17087.

RELNOTES[INC]: query --output=proto --order_output=deps now returns targets in topological order (previously there was no ordering).

PiperOrigin-RevId: 501548057
Change-Id: Ied86694fc6c7f9fc2243af1ce43bc778509cbe44

Co-authored-by: Googler <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 We'll consider working on this in future. (Assignee optional) team-Performance Issues for Performance teams type: bug
Projects
None yet
Development

No branches or pull requests

7 participants