[Datafactory] Add folders to Pipeline and Dataset #3636

arsunda · 2018-08-14T16:58:10Z

This checklist is used to make sure that common issues in a pull request are addressed. This will expedite the process of getting your pull request merged and avoid extra work on your part to fix issues discovered during the review process.

PR information

The title of the PR is clear and informative.
There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, see this page.
Except for special cases involving multiple contributors, the PR is started from a fork of the main repository, not a branch.
If applicable, the PR references the bug/issue that it fixes.
Swagger files are correctly named (e.g. the api-version in the path should match the api-version in the spec).

Quality of Swagger

I have read the contribution guidelines.
My spec meets the review criteria:
- The spec conforms to the Swagger 2.0 specification.
- The spec follows the guidelines described in the Swagger checklist document.
- Validation tools were run on swagger spec(s) and have all been fixed in this PR.

Related task: https://msdata.visualstudio.com/Azure%20Data%20Factory/_workitems/edit/277770

Adding folders to pipeline definition

Adding folders to Dataset definition

AutorestCI · 2018-08-14T16:59:38Z

Automation for azure-sdk-for-python

The initial PR has been merged into your service PR:
Azure/azure-sdk-for-python#2293

AutorestCI · 2018-08-14T17:02:32Z

Automation for azure-sdk-for-java

The initial PR has been merged into your service PR:
Azure/azure-sdk-for-java#2136

azuresdkci · 2018-08-14T17:15:40Z

Can one of the admins verify this patch?

marstr · 2018-08-14T17:40:33Z

...atafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/entityTypes/Dataset.json

@@ -37,6 +37,16 @@
          "items": {
            "type": "object"
          }
+        },
+        "folder": {
+          "description": "The folder that this Dataset is in.",


If this property is omitted, is there a default folder that is used? If so, it may be worth mentioning.

All the entities will appear at the root level unless a folder is specified. If that is worth mentioning then i can add that in

Your call :)

marstr · 2018-08-14T17:40:43Z

...atafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/entityTypes/Dataset.json

+          "type": "object",
+          "properties": {
+            "name": {
+              "description": "The name of the folder that this Dataset is in.",


AutorestCI · 2018-08-14T18:39:05Z

Automation for azure-sdk-for-ruby

Nothing to generate for azure-sdk-for-ruby

AutorestCI · 2018-08-14T18:43:28Z

Automation for azure-sdk-for-node

The initial PR has been merged into your service PR:
Azure/azure-sdk-for-node#3449

AutorestCI · 2018-08-14T18:46:37Z

Automation for azure-sdk-for-go

The initial PR has been merged into your service PR:
Azure/azure-sdk-for-go#2566

ravbhatnagar · 2018-08-15T07:36:54Z

This change should ideally go in a new api-version. In strongly typed SDKs, the property will be ignored. And the object which is returned from the GET is used for PUT, it can unintentionally override the state of the resource on the server and now the pipeline will be at root level?

arsunda · 2018-08-15T19:17:57Z

Folders is already being used, especially in the webapp/UI to organize the resources. Currently customers who have pipelines/datasets with folders and try to deploy them with the SDK loose their folder information. I don't think it needs to go into a new api version as it is not a introducing a new behavior but rather is helping customers get unblocked from their current experience

arsunda · 2018-08-17T01:04:11Z

@ravbhatnagar @marstr any pending comments/updates?

marstr · 2018-08-17T16:59:58Z

From my side, I'm waiting for @ravbhatnagar to remove the WaitForARMFeedback label to confirm whether or not he approves of adding new parameters inside of an API version. His concern about slicing should be taken seriously. :)

marstr · 2018-08-24T20:52:14Z

I'm going to be OOF all next week, so I'm going to have this PR reassigned.

ravbhatnagar · 2018-08-27T06:25:34Z

@arsunda When you say "already being used", do you mean that on the service side they are already implemented in the current api-version? Please confirm.
@SDK folks - If the answer to this question is YES, I leave to you folks to decide. It is technically still a breaking change. But its bring SDK behavior at par with service.

ravbhatnagar · 2018-08-27T06:53:23Z

Got confirmation from service team that this has existed on the service side. The change is only adding it to swagger now. Signing off from ARM side.

lmazuel

LGTM

arsunda added 2 commits August 14, 2018 09:49

Update Pipeline.json

b207da0

Adding folders to pipeline definition

Update Dataset.json

89afa6a

Adding folders to Dataset definition

AutorestCI mentioned this pull request Aug 14, 2018

[AutoPR datafactory/resource-manager] [Datafactory] Add folders to Pipeline and Dataset Azure/azure-sdk-for-python#3138

Merged

AutorestCI mentioned this pull request Aug 14, 2018

[AutoPR datafactory/resource-manager] [Datafactory] Add folders to Pipeline and Dataset Azure/azure-sdk-for-java#2323

Merged

azuresdkci assigned marstr Aug 14, 2018

azuresdkci requested a review from marstr August 14, 2018 17:21

marstr added the WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required label Aug 14, 2018

marstr requested a review from ravbhatnagar August 14, 2018 17:39

marstr reviewed Aug 14, 2018

View reviewed changes

arsunda added 2 commits August 14, 2018 11:38

Update Pipeline.json

e4a6b76

Update Dataset.json

c0f87ba

AutorestCI mentioned this pull request Aug 14, 2018

[AutoPR datafactory/resource-manager] [Datafactory] Add folders to Pipeline and Dataset Azure/azure-sdk-for-node#3338

Merged

AutorestCI mentioned this pull request Aug 14, 2018

[AutoPR datafactory/resource-manager] [Datafactory] Add folders to Pipeline and Dataset Azure/azure-sdk-for-go#2460

Merged

deata approved these changes Aug 14, 2018

View reviewed changes

marstr added the Reassign label Aug 24, 2018

marstr removed their assignment Aug 25, 2018

ravbhatnagar added the ARMSignedOff <valid label in PR review process>add this label when ARM approve updates after review label Aug 27, 2018

ravbhatnagar removed the WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required label Aug 27, 2018

hvermis assigned marstr Aug 28, 2018

lmazuel approved these changes Aug 28, 2018

View reviewed changes

lmazuel merged commit 8ff2bf7 into Azure:master Aug 28, 2018

haoyingli mentioned this pull request Sep 5, 2018

[data factory] add tumblingWindowTrigger dependsOn and add Folder to pipeline and da… Azure/azure-sdk-for-net#4741

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Datafactory] Add folders to Pipeline and Dataset #3636

[Datafactory] Add folders to Pipeline and Dataset #3636

arsunda commented Aug 14, 2018

AutorestCI commented Aug 14, 2018 •

edited

Loading

AutorestCI commented Aug 14, 2018 •

edited

Loading

azuresdkci commented Aug 14, 2018

marstr Aug 14, 2018

arsunda Aug 14, 2018

marstr Aug 14, 2018

arsunda Aug 14, 2018

marstr Aug 14, 2018

AutorestCI commented Aug 14, 2018 •

edited

Loading

AutorestCI commented Aug 14, 2018 •

edited

Loading

AutorestCI commented Aug 14, 2018 •

edited

Loading

ravbhatnagar commented Aug 15, 2018

arsunda commented Aug 15, 2018

arsunda commented Aug 17, 2018

marstr commented Aug 17, 2018

marstr commented Aug 24, 2018

ravbhatnagar commented Aug 27, 2018

ravbhatnagar commented Aug 27, 2018

lmazuel left a comment

[Datafactory] Add folders to Pipeline and Dataset #3636

[Datafactory] Add folders to Pipeline and Dataset #3636

Conversation

arsunda commented Aug 14, 2018

PR information

Quality of Swagger

AutorestCI commented Aug 14, 2018 • edited Loading

Automation for azure-sdk-for-python

AutorestCI commented Aug 14, 2018 • edited Loading

Automation for azure-sdk-for-java

azuresdkci commented Aug 14, 2018

marstr Aug 14, 2018

Choose a reason for hiding this comment

arsunda Aug 14, 2018

Choose a reason for hiding this comment

marstr Aug 14, 2018

Choose a reason for hiding this comment

arsunda Aug 14, 2018

Choose a reason for hiding this comment

marstr Aug 14, 2018

Choose a reason for hiding this comment

AutorestCI commented Aug 14, 2018 • edited Loading

Automation for azure-sdk-for-ruby

AutorestCI commented Aug 14, 2018 • edited Loading

Automation for azure-sdk-for-node

AutorestCI commented Aug 14, 2018 • edited Loading

Automation for azure-sdk-for-go

ravbhatnagar commented Aug 15, 2018

arsunda commented Aug 15, 2018

arsunda commented Aug 17, 2018

marstr commented Aug 17, 2018

marstr commented Aug 24, 2018

ravbhatnagar commented Aug 27, 2018

ravbhatnagar commented Aug 27, 2018

lmazuel left a comment

Choose a reason for hiding this comment

AutorestCI commented Aug 14, 2018 •

edited

Loading

AutorestCI commented Aug 14, 2018 •

edited

Loading

AutorestCI commented Aug 14, 2018 •

edited

Loading

AutorestCI commented Aug 14, 2018 •

edited

Loading

AutorestCI commented Aug 14, 2018 •

edited

Loading