Change inner dtypes of structs to tuple lists

[philss]

This change is necessary to keep the position of fields predictable.

This is related to #847

[maennchen]

@philss I expected the order of the dtype to be respected given that we provide it as a list now and not a map. But that doesn't seem to work.

I crafted the following tests to test my hypothesis:

Mix.install([
  {:explorer,
   github: "elixir-explorer/explorer", branch: "ps-change-struct-inner-dtypes-to-keylists"}
])

alias Explorer.DataFrame
alias Explorer.Series

ExUnit.configure(exclude: [:skip])
ExUnit.start(autorun: false)

defmodule DTypeTest do
  use ExUnit.Case

  test "preserves manually provided dtype order" do
    assert {:struct, [{"b", :string}, {"a", :string}]} =
             [%{"a" => "a", "b" => "b"}, %{"b" => "b", "a" => "a"}]
             |> Series.from_list(dtype: {:struct, [{"b", :string}, {"a", :string}]})
             |> then(& &1.dtype)

    assert {:struct, [{"a", :string}, {"b", :string}]} =
             [%{"a" => "a", "b" => "b"}, %{"b" => "b", "a" => "a"}]
             |> Series.from_list(dtype: {:struct, [{"a", :string}, {"b", :string}]})
             |> then(& &1.dtype)
  end

  test "can cast dtype order" do
    assert {:struct, [{"b", :string}, {"a", :string}]} =
             [%{"a" => "a", "b" => "b"}, %{"b" => "b", "a" => "a"}]
             |> Series.from_list(dtype: {:struct, [{"a", :string}, {"b", :string}]})
             |> Series.cast({:struct, [{"b", :string}, {"a", :string}]})
             |> then(& &1.dtype)
  end

  test "infers correctly ordered dtype from ordered source" do
    assert {:struct, [{"b", :string}, {"a", :string}]} =
             """
             {"col": {"b": "b", "a": "a"}}
             """
             |> DataFrame.load_ndjson!()
             |> then(& &1["col"].dtype)

    assert {:struct, [{"a", :string}, {"b", :string}]} =
             """
             {"col": {"a": "a", "b": "b"}}
             """
             |> DataFrame.load_ndjson!()
             |> then(& &1["col"].dtype)
  end
end

ExUnit.run()

Output:

$ elixir test.exs
Excluding tags: [:skip]



  1) test can cast dtype order (DTypeTest)
     test.exs:27
     match (=) failed
     code:  assert {:struct, [{"b", :string}, {"a", :string}]} =
              [%{"a" => "a", "b" => "b"}, %{"b" => "b", "a" => "a"}]
              |> Series.from_list(dtype: {:struct, [{"a", :string}, {"b", :string}]})
              |> Series.cast({:struct, [{"b", :string}, {"a", :string}]})
              |> then(& &1.dtype)
     left:  {:struct, [{"b", :string}, {"a", :string}]}
     right: {:struct, [{"a", :string}, {"b", :string}]}
     stacktrace:
       test.exs:28: (test)



  2) test infers correctly ordered dtype from ordered source (DTypeTest)
     test.exs:35
     match (=) failed
     code:  assert {:struct, [{"b", :string}, {"a", :string}]} =
              "{\"col\": {\"b\": \"b\", \"a\": \"a\"}}\n"
              |> DataFrame.load_ndjson!()
              |> then(& &1["col"].dtype)
     left:  {:struct, [{"b", :string}, {"a", :string}]}
     right: {:struct, [{"a", :string}, {"b", :string}]}
     stacktrace:
       test.exs:36: (test)



  3) test preserves manually provided dtype order (DTypeTest)
     test.exs:15
     match (=) failed
     code:  assert {:struct, [{"b", :string}, {"a", :string}]} =
              [%{"a" => "a", "b" => "b"}, %{"b" => "b", "a" => "a"}]
              |> Series.from_list(dtype: {:struct, [{"b", :string}, {"a", :string}]})
              |> then(& &1.dtype)
     left:  {:struct, [{"b", :string}, {"a", :string}]}
     right: {:struct, [{"a", :string}, {"b", :string}]}
     stacktrace:
       test.exs:16: (test)


Finished in 0.3 seconds (0.00s async, 0.3s sync)
3 tests, 3 failures

Randomized with seed 206478

[philss]

@maennchen yeah, I think some sorting I'm doing is not necessary. Let me try to fix it.

[philss]

@maennchen please try again. I think it is correct now :)

[maennchen]

@philss Looking good :)