-
Notifications
You must be signed in to change notification settings - Fork 296
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kubernetes: Failure when codecgen is run on anonymous type, but not on wrapping type. #100
Comments
Please write the error you get, and what you expect. |
What I get is: api.Policy{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, Predicates:[]api.PredicatePolicy(nil), Priorities:[]api.PriorityPolicy(nil)} So encoding/json correctly parses that input, however codec doesn't parse anything. |
To be clear:
|
Weird. I cannot reproduce this. I ran the test locally and it all works fine. I didn't test codecgen, but I tested codec with []byte and io.Reader, and compared against using encoding/json. I got similar outputs. I think there may be something environmental going on here. BTW, codec never falls back to encoding/json. codec works by first checking if the type implements codec.Selfer, and delegating to that. You can use codecgen to generate a static implementation of codec.Selfer (which fully bypasses reflection except where it sees an interface). BTW, you will usually know if you are using codecgen, because you will get much better than 2X (especially for encoding). My benchmarks show results of 2X over encoding/json in reflection-mode for encoding and decoding, and 13X and 3X in codecgen-mode for encoding and decoding.
|
@ugorji - did you generated the files? I'm reproducing it 100% times. |
I've added "ErrorIfNoField = true" and what I get now is:
source: |
I think I've got something. I had the following stacktrace:
It suggests, that Codec is trying to decode TypeMeta. However, there are no Typemeta-related fields in the input, so it should just parse predicated &pririties fields. |
@wojtek-1 your bug report and comments said you DID NOT run codecgen. But this stack trace is from codecgen. That's why I always ask you for your output and expected output. I've been looking at standard reflection+based mode (ie without codecgen generated Selfer implementation. Will run codecgen and look again. |
@ugorji - no i did NOT run codecgen |
or maybe to be more clear:
|
@wojtek-t Thanks. I can reproduce it now. I was previously running codecgen on go-get folder of github.com/kubernetes/kubernetes/pkg/api/ instead of k8s.io/kubernetes/pkg/api . The issue has to do with anonymous fields. I'm looking into it now. |
Awesome - thanks! |
FYI - I will be on vacation for a week from starting tomorrow and it's still not clear if someone picks it up from me or I will get back to it once I'm back. |
Ok. Well, let's try to wrap it up before you leave then ;) |
This effectively calls itself recursively and leads to an expected stackoverflow. To fix, check if the variable name == the constant variable name of the type in the Selfer implementation (x). If it is, then do not attempt to call the Selfer method. In addition, the following fixes were applied: When encoding extensions, use an address if the kind is not Struct or Array. This way, we don't copy unnecessarily. When encoding based on the known interfaces which may be implemented i.e. encoding.(Binary|Text)(M|Unm)arshaler, json.(M|Unm)arshaler, codec.Selfer, the value to encode may not be addressable but the implementation is on a pointer. In that situation, copy the value to an addressable value and call the implementation there. Updates #100
The problem was that you ran codecgen on a value (e.g. api.TypeMeta) which you then embedded, To fix, you MUST also run codecgen on the containing types, so that they have their own Note: You will have the same issue if you embed anything that is a The key thing to note here is that, when using anonymous fields, the methods are inherited. Please update the source and run codecgen on k8s.io/kubernetes/plugin/pkg/scheduler/api . |
yes - generating this file to the main types helps (also removing generated types for pkg/api/types.go helps). That's sad that it works like that, but I understand the explanation. Thanks a lot for help! |
@wojtek-t closing this as there isn't anything that can be done; this is the way "go" works. |
@wojtek-t Another thing to note: Given a type T, T's method set does not include methods bound to *T. So if you embed api.TypeMeta, then the Selfer methods are not "inherited". However, since you embed *api.TypeMeta, then its Selfer methods are "inherited". Go's embedded method inheritance is "wonky" :( and is good when it's useful, but it's bad at times like these :( |
To reproduce I'm using:
kubernetes/kubernetes@master...wojtek-t:use_ugorij#diff-22207499f9ab0d60831b9c388397f034R1
Input data:
kubernetes/kubernetes@master...wojtek-t:use_ugorij#diff-29f2371dad30835b344a6ca05b473eaeR1
I did NOT run codecgen for the file that contains definition of that jsons:
https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/scheduler/api/types.go
IIUC, this should result in falling back to encoding/json (which works fine). However this is not the case (i.e. the output is empty)
@ugorji
The text was updated successfully, but these errors were encountered: