Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change target to tables format for Multivariate regressor. #14

Merged
merged 6 commits into from
Mar 23, 2020

Conversation

ayush-1506
Copy link
Member

Fixes: #11

@ayush-1506 ayush-1506 changed the title Change target to tables format Change target to tables format for Multivariate regressor Mar 15, 2020
@ayush-1506 ayush-1506 requested a review from ablaom March 15, 2020 07:42
@ayush-1506
Copy link
Member Author

@ablaom I'm not sure why the test fails on travis; they pass locally. Any idea?

@@ -65,7 +65,7 @@ function collate(model::Union{NeuralNetworkRegressor, MultivariateNeuralNetworkR
row_batches = Base.Iterators.partition(1:length(y), batch_size)

Xmatrix = MLJModelInterface.matrix(X)'
if y isa AbstractVector{<:Tuple}
if y isa Tables.MatrixTable
return [(Xmatrix[:, b], ymatrix[:, b]) for b in row_batches]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be if Tables.istable(y) not if y isa Tables.MatrixTable, which is too restrictive. For, examlple, user is allowed to send a DataFrame.

src/regressor.jl Outdated
data = collate(model, X, y, model.batch_size)

target_is_multivariate = y isa AbstractVector{<:Tuple}
target_is_multivariate = y isa Tables.MatrixTable
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto here

@FluxML FluxML deleted a comment from ayush-1506 Mar 16, 2020
@ablaom
Copy link
Collaborator

ablaom commented Mar 16, 2020

Sorry about the n_output comment. My mistake.

That is weird. Your PR is also passing locally for me.

@ablaom
Copy link
Collaborator

ablaom commented Mar 16, 2020

Is this a clue? It seems travis is picking an ancient version of MLJBase for testing. We are up to 0.12.1 now, which is the version my local test used.

Screen Shot 2020-03-16 at 2 42 17 PM

@ayush-1506
Copy link
Member Author

I'll look into this. Not sure why it's picking up an older version. I could try pinning MLJBase to the required version, hopefully that should solve it. (Although I guess this shouldn't be the preferred method)

@ablaom
Copy link
Collaborator

ablaom commented Mar 16, 2020

FYI: As I understand it, [compat] declarations are ignored for packages not listed under [deps] (see JuliaLang/Pkg.jl#1179) (although this may change in the future (JuliaLang/Pkg.jl#1285)).

@ayush-1506
Copy link
Member Author

@ablaom At the moment, all packages in [deps] are also declared in [compat]. How is that related to MLJBase?

@ayush-1506
Copy link
Member Author

Also, what I'm unable to understand why travis passed on the previous PRs but fails for this. (I've not made any additional changes to Project.toml here, so it should naturally pull the same version of MLJBase (0.12.0) as in previous PRs, right?)

@ablaom
Copy link
Collaborator

ablaom commented Mar 18, 2020

@ablaom At the moment, all packages in [deps] are also declared in [compat]. How is that related to MLJBase?

For new release requests to be fast-tracked (auto-merged into General.jl) it is now a requirement that all [deps] not in the standard library be given [compat] upper bounds. As to your question, " How is that related to MLJBase?" I'm not sure I understand what you mean. Of course the fact that we cannot cap versions of "test-only" packages is a flaw, but this is a bigger issue, as I point out above, and should not concern us here: As I understand it, testing ought to load the latest version of [targets] packages not creating conflicts with the [deps]/[compat] specification. In the case of MLJBase, this should be 0.12.1 (or thereabouts) and I have no idea why travis is not using this version.

Also, what I'm unable to understand why travis passed on the previous PRs but fails for this. (

Sorry, me neither. Have you tried re-triggering travis with a trivial commit?

@ayush-1506 ayush-1506 changed the title Change target to tables format for Multivariate regressor Change target to tables format for Multivariate regressor. Mar 18, 2020
@ayush-1506
Copy link
Member Author

Just tried that, still fails. @tlienart would be great if you could also have a look when you get time. Maybe there's something we might have overlooked. Thanks

@tlienart
Copy link
Collaborator

wrong compat bound for Tables, will fix

@tlienart
Copy link
Collaborator

Uh I don't have write access...

Basically most important one is to bump the tables compat to ^1.0.

Notes: (at the risk of sounding annoying)

  • please use Juno's automatic formater to get rid of spurious whitelines and have more consistent code formatting
  • please add docstrings
  • please add comments apart for the super obvious places, assume someone else than you will have to maintain this

Also ideally please give Anthony and I write access (I'm assuming Anthony already has it).

Otherwise it's great that MLJFlux is back on track!

@ayush-1506
Copy link
Member Author

ayush-1506 commented Mar 18, 2020

@tlienart Thanks for that. Unfortunately travis still pulls the older MLJBase. Also, I don't have the access to add others as collaborators, I think @ablaom can give you the required access.

@tlienart
Copy link
Collaborator

Make it an explicit dependency, add compat bounds and see what fails on travis

@ablaom
Copy link
Collaborator

ablaom commented Mar 20, 2020

Given @tlienart admin access

@ayush-1506
Copy link
Member Author

@tlienart I tried a bunch of things, nothing seems to work. (Even locally, I deleted the manifest file and did a clean install and it pulls the older MLJBase). Not quite sure what's going wrong. Would be great if you could take a look too. Thanks.

@ablaom
Copy link
Collaborator

ablaom commented Mar 23, 2020

Okay, finally we are getting somewhere. By retroactively adding upper bounds to all versions of all MLJ repos, we get useful conflict error from MLJFlux travis test, suggesting MLJScientifictTypes is the culprit: It needs to update its ColorTypes [compat], which I am now working on.

JuliaRegistries/General#11382

@ayush-1506 ayush-1506 merged commit 0fdb7ed into master Mar 23, 2020
@ablaom ablaom deleted the multitask branch July 6, 2020 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Multivariate (multitask) format needs attention?
3 participants