Refactor repo root dir layout

[konrad-jamrozik]

Per following @weshaggard's comments:

• Add global.json and other missing paths to CODEOWNERS #5044 (comment)
• Add global.json and other missing paths to CODEOWNERS #5044 (comment)
• Add global.json and other missing paths to CODEOWNERS #5044 (comment)
• Add global.json and other missing paths to CODEOWNERS #5044 (comment)

we should restructure our repo directory layout. Namely, we should:

• Move /.azure-pipelines/ under /eng/pipelines/
• Move /config/ under /eng/
• Move /scripts/ under /eng/
• Move /src/ and /web/ under appropriate subdirs of /tools/

[konrad-jamrozik]

@weshaggard I have few question about files under /.azure-pipelines/. There are pipelines defined over the .yml files in that dir:

.yml	pipeline
.azure-pipelines/apiview-sync-staging.yml	APIView Sync Production to Staging
.azure-pipelines/apiview.yml	APIView - Deploy - Staging
.azure-pipelines/apiview.yml	APIView
.azure-pipelines/warden.yml	azure-sdk-tools - warden

My questions:

• Is there an easy, reliable way for me to ensure the list above is exhaustive? I am fairly sure I should be able to find relevant ADO API call to return me json with all the pipelines and so search for given .yml, (I mean, our notification-configuration does something similar) but I was just wondering if we already have some automation for that.
• How these pipelines came to be? Our pipeline-generator and prepare-pipelines.yml seems to create only pipelines that match inheritors of PipelineConvention, but these pipelines do not seem to match any convention? Have they been created by hand?
• Is it a mistake that there are two pipelines based off .azure-pipelines/apiview.yml ? I compared them and I don't see any meaningful difference. Stages, params, vars, they all seem semantically equivalent.
• I am thinking about doing simple migration as follows: move these build definitions .yml files to /eng/pipelines/, and as soon as I merge the PR, go to all these pipelines and point them to the new .yml file locations. This means they will be down for few seconds, but I suppose this is not a problem. Does that sound good?

[weshaggard]

1. See https://dev.azure.com/azure-sdk/internal/_git/azure-sdk-devops we dump the definitions there from time to time and you could grep over them there and even refresh if you need to.
2. These pipelines and most of them in the tools repo was created manually as the don't follow the patterns of the mono repos.
3. No we often will configure different pipelines by share the same yml. Usually in these cases there will be variables or branches or such created for the different pipelines.
4. Yes I'm good with migrating these but lets please make sure the apiview team is aware of it.

[konrad-jamrozik]

@praveenkuttappan @tjprescott @chidozieononiwu per this comment above, I am going to move the source .yml files for the listed API view pipelines.

These pipelines may be broken for few minutes max as I do that.

Can you confirm this works with you?

Also: can you tell me why we need both APIView - Deploy - Staging and APIView ? They use the same build definition and upon inspection, I was unable to determine what is the difference between these two.

[tjprescott]

I'll defer to @praveenkuttappan on these APIView pipelines, as I don't think those are the ones for Python, Swift or Cadl.

[praveenkuttappan]

@praveenkuttappan @tjprescott @chidozieononiwu per this comment above, I am going to move the source .yml files for the listed API view pipelines.

These pipelines may be broken for few minutes max as I do that.

Can you confirm this works with you?

Also: can you tell me why we need both APIView - Deploy - Staging and APIView ? They use the same build definition and upon inspection, I was unable to determine what is the difference between these two.

I had this in my TODO list to move them to APIView directory. We needed two pipelines to keep production deployment and dev/preprod deployment builds completely separated and releases to those environments are based on these pipelines accordingly. We can deploy all environments from same pipeline but then this can be more problematic to error prone to accidentally deploy from a wrong pipeline run to production slot.

[praveenkuttappan]

@konrad-jamrozik It's totally fine to move the pipeline yaml. I can submit a PR since I need to make trigger change as well.

[praveenkuttappan]

@konrad-jamrozik I have submitted PR #5244 to move pipeline YAMLs

[konrad-jamrozik]

@weshaggard @praveenkuttappan regarding /src/, appears to me almost everything under there is APIView.

Should we thus make a move like /src/ -> /tools/APIView/ ?

I am also curious: what dependencies APIView has on the rest of the repository? In other words, what would break if we would move APIView into its own repo?

The only other two things I see there besides APIView are:

• /src/dotnet/Azure.ClientSDK.Analyzers/, which I guess should not be moved to /tools/ as it is not a tool but a package. Also, I see APIView depends on it.

• /src/dotnet/Mgmt.CI.BuildTools/, which is hard for me to determine what it is or what depends on it. I guess these are some custom MSBuild tasks? In any case, likely also shouldn't be moved to /tools/.

[weshaggard]

Yes I would like to eventually move everything under src to somewhere under tools. While some of these things aren't stand alone tools they still fit in the general "tools" category and as such I would like them moved under the tools directory.

• tools/apiview - Please work with @praveenkuttappan and @maririos with this move to try and minimize disruption.
• tools/dotnet-analyzers - This is the .NET tools analyzer
• tools/dotnet-mgmt-tools - This is an old set of tools that is still used for the track 1 mgmt package build. It isn't updated much currently but I don't think we can remove it fully yet.

[konrad-jamrozik]

I have created an issue to track the move of /src/:

• Move contents of /src/ to /tools/ #5641

I have completed all the other moves, so closing this work item.