Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deserializer-properties are not propagated in delegating calls for serde creation #269

Open
DarrenBishop opened this issue Aug 4, 2020 · 1 comment

Comments

@DarrenBishop
Copy link

The call-site

    (jsa/serde reg
               {:avro.schema-registry/url schema-registry-url
                :avro.schema-registry/client schema-registry-client}
               {:avro/schema schema
                :key? key?
                :avro/coercion-cache coercion-cache
                :read-only? read-only?})

and the call-target

(defn serde
  "Given a type and logical type registry, a schema registry config with
  either a client or a URL and an Avro topic descriptor, build and
  return a Serde instance."
  [type-registry
   {:keys [avro.schema-registry/client
           avro.schema-registry/url]
    :as   registry-config}
   {:keys [avro/schema
           avro/coercion-cache
           key?
           deserializer-properties
           read-only?]
    :as   topic-config}]

Later in the code there are checks against property presence/values that cannot be true as a result; for example:

(let [avro-data (if (get deserializer-properties "specific.avro.reader")
                                                    (.deserialize base-deserializer ^String topic #^bytes raw-data ^Schema avro-schema)
                                                    (.deserialize base-deserializer ^String topic #^bytes raw-data))]

I am new to Clojure and this library, so hope my assessment is sound (and useful).

@cddr
Copy link
Contributor

cddr commented Aug 5, 2020

I see what you mean. I think this would be a fairly easy fix if you want to have a shot at submitting a PR.

Having said that, if you're not using this feature yet, I'd give some thought to using io.confluent.kafka.streams.serdes.avro.GenericAvroSerde in combination with https://github.com/cddr/edn-avro. The readme of that project explains some of the mistakes I think we made with the avro serde in jackdaw and it attempts to solve the problem of marshalling data from clojure into avro and back in a way that is simpler, more interoperable with built-in Kafka tooling, and doesn't get in the way when you want to do more advanced things.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants