Support Postgrex v0.14.0 #141

slashdotdash · 2018-11-06T15:17:07Z

Attempting to compile EventStore with the latest version of Postgrex published to Hex (v0.14.0) fails with the following compile error:

Generated postgrex app
==> eventstore
Compiling 49 files (.ex)

== Compilation error in file lib/event_store/storage/reader.ex ==
** (CompileError) lib/event_store/storage/reader.ex:73: Postgrex.Timestamp.__struct__/0 is undefined, cannot expand struct Postgrex.Timestamp
    lib/event_store/storage/reader.ex:73: (module)
could not compile dependency :eventstore, "mix compile" failed. You can recompile this dependency with "mix deps.compile eventstore", update it with "mix deps.update eventstore" or clean it with "mix deps.clean eventstore"

This is because Postgrex.Timestamp has been deprecated and removed from Postgrex which is used in EventStore.Storage.Reader. The usage must be made optional (e.g. Code.ensure_loaded?(Postgrex.Timestamp)) to support both v0.13.0 and older and the latest v0.14.0.

The text was updated successfully, but these errors were encountered:

slashdotdash · 2018-11-11T20:55:26Z

Postgrex v0.14.0 expects the JSON encoder to provide an encode_to_iodata!/1 function. This function has been removed from Poison in v4.x which means that Poison cannot be directly used with the latest Postgrex.

This is needed for EventStore to support the jsonb data type (for event data & metadata).

vasspilka · 2018-11-12T07:40:42Z

@slashdotdash What would you consider as a valid change for this? Would maybe a migration to Jason + a version bump to 0.16 be accepted as a solution? Or maybe just drop support for Poison 4? Jason does not have the as: but one can use struct/2 to achieve the same result.

slashdotdash · 2018-11-12T10:55:16Z

As both Postgrex and Phoenix have migrated from Poison to Jason for JSON encoding then EventStore can do the same. In which case we should deprecate support for Poison and previous Postgrex versions (v0.13.0 and older).

This will also impact Commanded since it uses Poison for event data & metadata deserialization (in Commanded.Serialization.JsonSerializer).

The Jason library outlines its differences from Poison, namely:

Jason follows the JSON spec more strictly, for example it does not allow unescaped newline characters in JSON strings - e.g. "\"\n\"" will produce a decoding error.
no support for decoding into data structures (the as: option).
no built-in encoders for MapSet, Range and Stream.
no support for encoding arbitrary structs - explicit implementation of the Jason.Encoder protocol is always required.
different pretty-printing customisation options (default pretty: true works the same)

Not being able to natively encode arbitrary structs may cause issues since Poison does not require a custom encoder for nested structs, but Jason does.

Ideally I'd like EventStore to support middleware for writing/reading events allowing you to specify your own serializer and any additional event processing (e.g. encryption, enrichment, upcasting, versioning, auditing, etc.).

vasspilka · 2018-11-12T11:06:35Z

Yes, I am aware, I already have written a Jason serializer, so when I get the time I'll look into these issues and send a PR 💙 💜 ❤️

raarts · 2018-11-30T14:42:53Z

Hi, just trying out Commanded, and experiencing this problem. What is the recommended workaround, if I can't wait for this PR to be merged and a new release been published? Thanks.

slashdotdash · 2018-11-30T14:48:36Z

@raarts Unfortunately there is no workaround since Postgrex v0.14.0 has broken the integration with Poison for JSON serialization. We need to migrate EventStore to use Jason.

raarts · 2018-11-30T15:01:15Z

None at all? Maybe I can go back to an older Postgrex version (I am using phoenix 1.4 though)

slashdotdash · 2018-11-30T15:02:44Z

Phoenix doesn't have a dependency on Postgrex. I'm assuming you are also using Ecto v3 which does depend upon the latest Postgrex release. You could try using Ecto v2.2.x instead for now.

raarts · 2018-12-01T09:07:40Z

@slashdotdash I tried but got stuck in compile errors/dependency problems. I'll have to wait. Any timeline?

raarts · 2018-12-05T09:57:23Z

Just in case someone else has the same problem, this worked:

mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new-1.3.4.ez
mix phx.new eventsourcing
mix archive.install hex phx_new 1.4.0

(Did not find a way to keep two different versions of that mix task on my machine and pick between them)

vasspilka · 2018-12-05T10:25:16Z

It would have been enough to use postrex 1.3 in your 1.4 Phoenix app

…

On Wed, Dec 5, 2018, 10:57 Ron Arts ***@***.*** wrote: Just in case someone else has the same problem, this worked: mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new-1.3.4.ez mix phx.new eventsourcing mix archive.install hex phx_new 1.4.0 (Did not find a way to keep two different versions of that mix task on my machine and pick between them) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#141 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADH2sypSp7GgZ1R5akF_OUIePSvyHBg1ks5u15iGgaJpZM4YQpsP> .

raarts · 2018-12-05T12:16:16Z

Well, I just tried again, but:

Failed to use "postgrex" because
  apps/api/mix.exs requires ~> 0.13.0
  ecto_sql (version 3.0.0) requires ~> 0.14.0

Adding override: true to the postgrex dependency resulted in:

Failed to use "db_connection" (versions 2.0.0 to 2.0.3) because
  ecto_sql (version 3.0.0) requires ~> 2.0
  postgrex (versions 0.13.0 to 0.13.5) requires ~> 1.1

Next I changed the mix.exs file in deps/ecto_sql to require 1.0 with override, but running mix deps.get after that change did not seem to pick that up, was still complaining about requiring 2.0. Tried editing mix.lock file, but also didn't get picked up, and then I was lost.

I remember an earlier attempt ending up in compile errors. So I guess it's not as easy as it looks.

vasspilka · 2018-12-06T11:54:04Z

Yes, yes, its Ecto 3 causing that not much to do with phoenix as slashdot noted. We are working on a migration but seems no one really has the time for it at the moment..

…

On Wed, 5 Dec 2018 at 13:16, Ron Arts ***@***.***> wrote: Well, I just tried again, but: Failed to use "postgrex" because apps/api/mix.exs requires ~> 0.13.0 ecto_sql (version 3.0.0) requires ~> 0.14.0 Adding override: true to the postgrex dependency resulted in: Failed to use "db_connection" (versions 2.0.0 to 2.0.3) because ecto_sql (version 3.0.0) requires ~> 2.0 postgrex (versions 0.13.0 to 0.13.5) requires ~> 1.1 Next I changed the mix.exs file in deps/ecto_sql to require 1.0 with override, but running mix deps.get after that change did not seem to pick that up, was still complaining about requiring 2.0. Tried editing mix.lock file, but also didn't get picked up, and then I was lost. I remember an earlier attempt ending up in compile errors. So I guess it's not as easy as it looks. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#141 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADH2s8dIVT9J6Dx2HYaDRXR-3HKpBWm3ks5u17kRgaJpZM4YQpsP> .

vasspilka · 2018-12-06T11:57:33Z

There is no other ecto_sql other than the v 3 one. Before that it was just ecto, with version 3 it got split into 2 dependencies. You can find more here http://blog.plataformatec.com.br/2018/10/a-sneak-peek-at-ecto-3-0-breaking-changes/.

darwinphi · 2018-12-09T11:35:44Z

What's the solution for this?

slashdotdash · 2018-12-09T12:33:27Z

The blocking issue is to migrate JSON serialization from Poison to Jason. This is a significant and breaking change which needs to be carefully implemented. Unfortunately I don’t have time right now to do so.

Can you continue using Postgrex v0.13.x for now?

davich · 2018-12-13T04:07:54Z

We're continuing to use Phoenix 1.3 and Postgrex 0.13 for now. Eagerly awaiting an update :) Thanks

theevangelista · 2018-12-29T15:41:47Z

Still can't compile on a new project:

== Compilation error in file lib/event_store/storage/reader.ex ==
** (CompileError) lib/event_store/storage/reader.ex:73: Postgrex.Timestamp.__struct__/0 is undefined, cannot expand struct Postgrex.Timestamp
    lib/event_store/storage/reader.ex:73: (module)
could not compile dependency :eventstore, "mix compile" failed. You can recompile this dependency with "mix deps.compile eventstore", update it with "mix deps.update eventstore" or clean it with "mix deps.clean eventstore"

Deps:

$ mix deps
* connection 1.0.4 (Hex package) (mix)
  locked at 1.0.4 (connection) a1cae722
  ok
* gen_stage 0.14.1 (Hex package) (mix)
  locked at 0.14.1 (gen_stage) 9d46723f
  ok
* poolboy 1.5.1 (Hex package) (rebar)
  locked at 1.5.1 (poolboy) 6b461639
  ok
* decimal 1.6.0 (Hex package) (mix)
  locked at 1.6.0 (decimal) bfd84d90
  ok
* fsm 0.3.1 (Hex package) (mix)
  locked at 0.3.1 (fsm) 087aa9b0
  ok
* db_connection 2.0.3 (Hex package) (mix)
  locked at 2.0.3 (db_connection) b4e8aa43
  ok
* uuid 1.1.8 (Hex package) (mix)
  locked at 1.1.8 (uuid) e22fc044
  ok
* postgrex 0.14.1 (Hex package) (mix)
  locked at 0.14.1 (postgrex) 63247d4a
  ok
* eventstore (Hex package) (mix)
  locked at 0.15.1 (eventstore) dfa07a11
  the dependency build is outdated, please run "mix deps.compile"

Elixir: 1.7.4 Compiled with OTP 21.2

slashdotdash · 2018-12-30T19:43:38Z

@sleepiejohn You need to use Postgrex v0.13.5

lewisf · 2019-01-03T03:21:09Z

Even though everyone has different setups, hoping people might find this helpful.

Here are the packages I had to downgrade (running multiple phoenix 1.4 apps in umbrella):

removed ecto_sql 3.0
downgraded ecto 3.0 to ecto 2.2
downgraded phoenix_ecto 4.0 to phoenix_ecto 3.6
downgraded postgrex 0.14.0 to postgrex 0.13.5
downgraded guardian_db from 2.0 to guardian_db 1.1 (2.0 relied on ecto 3.0)

Adzz · 2019-02-13T14:35:15Z

Am I right in thinking this is done now? Looks like there are commits in master switching from poison to Jason

slashdotdash · 2019-02-13T14:36:37Z

Yes, it's done in #143 and included in the v0.16.0 release published to Hex.

lewisf mentioned this issue Nov 7, 2018

Support DBConnection 2.0 #142

Closed

vasspilka mentioned this issue Nov 10, 2018

Update to Postgres 0.14 #143

Merged

slashdotdash closed this as completed Feb 13, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support Postgrex v0.14.0 #141

Support Postgrex v0.14.0 #141

slashdotdash commented Nov 6, 2018 •

edited

Loading

slashdotdash commented Nov 11, 2018

vasspilka commented Nov 12, 2018

slashdotdash commented Nov 12, 2018 •

edited

Loading

vasspilka commented Nov 12, 2018

raarts commented Nov 30, 2018 •

edited

Loading

slashdotdash commented Nov 30, 2018

raarts commented Nov 30, 2018

slashdotdash commented Nov 30, 2018 •

edited

Loading

raarts commented Dec 1, 2018

raarts commented Dec 5, 2018

vasspilka commented Dec 5, 2018 via email

raarts commented Dec 5, 2018

vasspilka commented Dec 6, 2018 via email

vasspilka commented Dec 6, 2018

darwinphi commented Dec 9, 2018

slashdotdash commented Dec 9, 2018

davich commented Dec 13, 2018

theevangelista commented Dec 29, 2018

slashdotdash commented Dec 30, 2018

lewisf commented Jan 3, 2019 •

edited

Loading

Adzz commented Feb 13, 2019

slashdotdash commented Feb 13, 2019

Support Postgrex v0.14.0 #141

Support Postgrex v0.14.0 #141

Comments

slashdotdash commented Nov 6, 2018 • edited Loading

slashdotdash commented Nov 11, 2018

vasspilka commented Nov 12, 2018

slashdotdash commented Nov 12, 2018 • edited Loading

vasspilka commented Nov 12, 2018

raarts commented Nov 30, 2018 • edited Loading

slashdotdash commented Nov 30, 2018

raarts commented Nov 30, 2018

slashdotdash commented Nov 30, 2018 • edited Loading

raarts commented Dec 1, 2018

raarts commented Dec 5, 2018

vasspilka commented Dec 5, 2018 via email

raarts commented Dec 5, 2018

vasspilka commented Dec 6, 2018 via email

vasspilka commented Dec 6, 2018

darwinphi commented Dec 9, 2018

slashdotdash commented Dec 9, 2018

davich commented Dec 13, 2018

theevangelista commented Dec 29, 2018

slashdotdash commented Dec 30, 2018

lewisf commented Jan 3, 2019 • edited Loading

Adzz commented Feb 13, 2019

slashdotdash commented Feb 13, 2019

slashdotdash commented Nov 6, 2018 •

edited

Loading

slashdotdash commented Nov 12, 2018 •

edited

Loading

raarts commented Nov 30, 2018 •

edited

Loading

slashdotdash commented Nov 30, 2018 •

edited

Loading

lewisf commented Jan 3, 2019 •

edited

Loading