Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/rdmp 73 cohort holdouts #1653

Merged
merged 30 commits into from
Nov 9, 2023
Merged

Conversation

JFriel
Copy link
Collaborator

@JFriel JFriel commented Oct 17, 2023

Allows for creation of holdout cohorts, they are generated via the cohort builder and the identifiers are stored in a catalogue

Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL found more than 10 potential problems in the proposed changes. Check the Files changed tab for more details.

@JFriel JFriel requested a review from jas88 October 20, 2023 12:56
@JFriel JFriel marked this pull request as ready for review October 20, 2023 12:56
/// <summary>
/// Required designer variable.
/// </summary>
private System.ComponentModel.IContainer components = null;

Check notice

Code scanning / CodeQL

Missed 'readonly' opportunity Note

Field 'components' can be 'readonly'.
DateTime.TryParseExact(minDate, "DD/MM/YYYY", new CultureInfo("en-GB"), DateTimeStyles.None, out _MinDate);
MinDate = _MinDate;
DateTime _MaxDate;
DateTime.TryParseExact(maxDate, "DD/MM/YYYY", new CultureInfo("en-GB"), DateTimeStyles.None, out _MaxDate);
Copy link
Member

@jas88 jas88 Oct 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor C# tip for the future: you can do that inline as "out var foo" instead of "DateTime foo; ... out foo", and you can write straight to the target instead of doing "out var tmp; foo = tmp;" (if it's a variable not a property!)


if (!string.IsNullOrWhiteSpace(cic.Description))
ui.CohortDescription = $"{cic.Description} ({Environment.UserName} - {DateTime.Now})";
return ui.ShowDialog() == DialogResult.OK ? ui.Result : ui.Result;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking this should be ui.Result:null like the method above, instead of returning the result regardless of the user action?

Comment on lines +77 to +80
catch (Exception e)
{
GlobalError("Unable to access datatable",e);
}

Check notice

Code scanning / CodeQL

Generic catch clause Note

Generic catch clause.
@jas88 jas88 merged commit adb2a51 into develop Nov 9, 2023
4 checks passed
@jas88 jas88 deleted the feature/RDMP-73-cohort-holdouts branch November 9, 2023 19:44
JFriel added a commit that referenced this pull request Dec 13, 2023
* Bump YamlDotNet from 13.3.1 to 13.4.0

Bumps [YamlDotNet](https://github.com/aaubry/YamlDotNet) from 13.3.1 to 13.4.0.
- [Release notes](https://github.com/aaubry/YamlDotNet/releases)
- [Commits](aaubry/YamlDotNet@v13.3.1...v13.4.0)

---
updated-dependencies:
- dependency-name: YamlDotNet
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump shogo82148/actions-setup-perl from 1.23.1 to 1.24.1

Bumps [shogo82148/actions-setup-perl](https://github.com/shogo82148/actions-setup-perl) from 1.23.1 to 1.24.1.
- [Release notes](https://github.com/shogo82148/actions-setup-perl/releases)
- [Commits](shogo82148/actions-setup-perl@v1.23.1...v1.24.1)

---
updated-dependencies:
- dependency-name: shogo82148/actions-setup-perl
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* fix checkbox issue

* improve confirmation text (#1639)

* improve confirmation text
* Loop tidyup, use var where possible

---------

Co-authored-by: jas88 <[email protected]>

* correct typo in create logging sql (#1640)

* Feature/ci codescan (#1641)

* Move SecurityCodescan.VS2019 to run on Github CI alone, integrate results with CodeQL
* Remove SecurityCodescan from Packages.md, no longer used via Nuget

---------

Co-authored-by: James A Sutherland <[email protected]>

* hide source control when not available

* Remove old Plugin object bits, tidy up (#1636)

* Remove old Plugin object bits, tidy up

* Purge remaining bits of AllExpiredPluginsNode

* Fix plugin display name in tree

* Update CreateNewDataExtractionProjectUI.cs

Casting fix

* Feature/rdmp42 delete plugins (#1642)

* add ui plugin delete functionality

* Warning and inheritance redundancy cleanups

* Narrow scope of catch clause per CodeQL warning

* Tidy Plugin name retrieval

---------

Co-authored-by: James A Sutherland <>
Co-authored-by: James Friel <[email protected]>

* Bump MongoDB.Driver from 2.21.0 to 2.22.0

Bumps [MongoDB.Driver](https://github.com/mongodb/mongo-csharp-driver) from 2.21.0 to 2.22.0.
- [Release notes](https://github.com/mongodb/mongo-csharp-driver/releases)
- [Commits](mongodb/mongo-csharp-driver@v2.21.0...v2.22.0)

---
updated-dependencies:
- dependency-name: MongoDB.Driver
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump YamlDotNet from 13.4.0 to 13.5.2 (#1644)

Bumps [YamlDotNet](https://github.com/aaubry/YamlDotNet) from 13.4.0 to 13.5.2.
- [Release notes](https://github.com/aaubry/YamlDotNet/releases)
- [Commits](aaubry/YamlDotNet@v13.4.0...v13.5.2)

---
updated-dependencies:
- dependency-name: YamlDotNet
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump YamlDotNet from 13.5.2 to 13.7.0 (#1647)

* Bump SSH.NET from 2020.0.2 to 2023.0.0 (#1646)

* Bump YamlDotNet from 13.7.0 to 13.7.1

Bumps [YamlDotNet](https://github.com/aaubry/YamlDotNet) from 13.7.0 to 13.7.1.
- [Release notes](https://github.com/aaubry/YamlDotNet/releases)
- [Commits](aaubry/YamlDotNet@v13.7.0...v13.7.1)

---
updated-dependencies:
- dependency-name: YamlDotNet
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump NLog from 5.2.4 to 5.2.5

Bumps [NLog](https://github.com/NLog/NLog) from 5.2.4 to 5.2.5.
- [Release notes](https://github.com/NLog/NLog/releases)
- [Changelog](https://github.com/NLog/NLog/blob/dev/CHANGELOG.md)
- [Commits](NLog/NLog@v5.2.4...v5.2.5)

---
updated-dependencies:
- dependency-name: NLog
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* New FTP/FTPS support, improve SFTP (#1649)

Co-authored-by: James Friel <[email protected]>

* Cut Windows exe file counts (#1650)

* Cut Windows exe file counts

* Remove manual deletion of files dotnet publish used to create

* Fix up docs links

* More doc link fixups

* Fix FAnsiSql Implementations README links

* Make GUI and CLI Windows exes single-file, update MSI accordingly

* Build source zip early on for inclusion later

---------

Co-authored-by: James Friel <[email protected]>
Co-authored-by: James A Sutherland <>

* Bump actions/setup-node from 3.8.1 to 4.0.0

Bumps [actions/setup-node](https://github.com/actions/setup-node) from 3.8.1 to 4.0.0.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](actions/setup-node@v3.8.1...v4.0.0)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

* Task/rdmp-32 Fix Progress Log writing using excessive resources (#1657)

* add load

* readd sorting

* revert sorting

* tidy up code

* Task/rdmp 32 Name DataTable Chunks (#1658)

* add chunk naming
* bump to 8.1.1-r1
* Add heading for WIP 8.1.1 release

---------

Co-authored-by: James A Sutherland <[email protected]>

* RDMP-15 Use .bak files as Data Loads (#1656)

* basic ui triggers
* working dl

---------

Co-authored-by: James A Sutherland <[email protected]>

* Bump FluentFTP from 48.0.1 to 48.0.3

Bumps [FluentFTP](https://github.com/robinrodricks/FluentFTP) from 48.0.1 to 48.0.3.
- [Release notes](https://github.com/robinrodricks/FluentFTP/releases)
- [Changelog](https://github.com/robinrodricks/FluentFTP/blob/master/RELEASES.md)
- [Commits](https://github.com/robinrodricks/FluentFTP/commits)

---
updated-dependencies:
- dependency-name: FluentFTP
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Feature/cleanup (#1659)

* Use var

* Whitespace fixes

* Remove stray parentheses

* Update SingletonNode.cs

Fix confused syntax, use Equ for simplicity

* Typos

Fix typos, also make some verbatim strings raw strings for better formatting.

* Spelling/grammar, wrongly marked partial classes

* Update SingletonNode.cs

Fix up equality test to be strict about types

* Fix typos and tidy tooltip cache logic

* Bump shogo82148/actions-setup-perl from 1.24.1 to 1.24.2

---
updated-dependencies:
- dependency-name: shogo82148/actions-setup-perl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump SixLabors.ImageSharp.Drawing from 2.0.0 to 2.0.1

Bumps [SixLabors.ImageSharp.Drawing](https://github.com/SixLabors/ImageSharp.Drawing) from 2.0.0 to 2.0.1.
- [Release notes](https://github.com/SixLabors/ImageSharp.Drawing/releases)
- [Commits](SixLabors/ImageSharp.Drawing@v2.0.0...v2.0.1)

---
updated-dependencies:
- dependency-name: SixLabors.ImageSharp.Drawing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump Terminal.Gui from 1.14.0 to 1.14.1 (#1662)

Bumps [Terminal.Gui](https://github.com/gui-cs/Terminal.Gui) from 1.14.0 to 1.14.1.
- [Release notes](https://github.com/gui-cs/Terminal.Gui/releases)
- [Commits](gui-cs/Terminal.Gui@v1.14.0...v1.14.1)

---
updated-dependencies:
- dependency-name: Terminal.Gui
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Restore buggy Equals behaviour on SingletonNode since some UI code relied on the bug

* Bump NUnit from 3.13.3 to 3.14.0

Bumps [NUnit](https://github.com/nunit/nunit) from 3.13.3 to 3.14.0.
- [Release notes](https://github.com/nunit/nunit/releases)
- [Changelog](https://github.com/nunit/nunit/blob/master/CHANGES.md)
- [Commits](nunit/nunit@v3.13.3...v3.14.0)

---
updated-dependencies:
- dependency-name: NUnit
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Fix Extraction Refresh Issue (#1671)

* fix extraction refresh issue
* reload, but not hot

* Bump Microsoft.NET.Test.Sdk from 17.7.2 to 17.8.0

Bumps [Microsoft.NET.Test.Sdk](https://github.com/microsoft/vstest) from 17.7.2 to 17.8.0.
- [Release notes](https://github.com/microsoft/vstest/releases)
- [Changelog](https://github.com/microsoft/vstest/blob/main/docs/releases.md)
- [Commits](microsoft/vstest@v17.7.2...v17.8.0)

---
updated-dependencies:
- dependency-name: Microsoft.NET.Test.Sdk
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Feature/rdmp 73 cohort holdouts (#1653)

* add top to cohort

* working holdout flow

* interim

* basic ui  flow

* now filtering

* working flow

* working auto-holdout

* fix test

* add query

* improved holdout

* tidy up code

* add description

* revert test db

* add holdout description

* add todo

* fixups from codeql

* Minor syntax fix

* Fix possible null deref

* fix todo url

* Tidy, typo fix

* Remove disused field

---------

Co-authored-by: James A Sutherland <[email protected]>
Co-authored-by: James A Sutherland <>

* RDMP-108 Unified UI Alerting from Processing Chunks (#1663)

* batched ui

* tidy up

* attempt to fix codeql

* fix linting

* add if check

* fix test

* add pragma

* fix up test

* update from codeql

* Update DataFlowPipelineEngine.cs

Tidy thread logic, fix double-disposal of objects

* Put chunk disposal in a finally block to avoid leaks

---------

Co-authored-by: James A Sutherland <[email protected]>
Co-authored-by: James A Sutherland <>

* Bump NLog from 5.2.5 to 5.2.6 (#1679)

Bumps [NLog](https://github.com/NLog/NLog) from 5.2.5 to 5.2.6.
- [Release notes](https://github.com/NLog/NLog/releases)
- [Changelog](https://github.com/NLog/NLog/blob/dev/CHANGELOG.md)
- [Commits](NLog/NLog@v5.2.5...v5.2.6)

---
updated-dependencies:
- dependency-name: NLog
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump Microsoft.SourceLink.GitHub from 1.1.1 to 8.0.0

Bumps [Microsoft.SourceLink.GitHub](https://github.com/dotnet/sourcelink) from 1.1.1 to 8.0.0.
- [Release notes](https://github.com/dotnet/sourcelink/releases)
- [Commits](https://github.com/dotnet/sourcelink/commits)

---
updated-dependencies:
- dependency-name: Microsoft.SourceLink.GitHub
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump Microsoft.XmlSerializer.Generator from 7.0.0 to 8.0.0

Bumps [Microsoft.XmlSerializer.Generator](https://github.com/dotnet/runtime) from 7.0.0 to 8.0.0.
- [Release notes](https://github.com/dotnet/runtime/releases)
- [Commits](dotnet/runtime@v7.0.0...v8.0.0)

---
updated-dependencies:
- dependency-name: Microsoft.XmlSerializer.Generator
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

* fix bump

* Fix coverage reporting (#1683)

* Rearrange lcov file handling
* Update coveralls action
* Point Github-Actions Dependabot updates at develop branch, add JFriel as reviewer

* Bump Spectre.Console from 0.47.0 to 0.48.0

Bumps [Spectre.Console](https://github.com/spectreconsole/spectre.console) from 0.47.0 to 0.48.0.
- [Release notes](https://github.com/spectreconsole/spectre.console/releases)
- [Commits](spectreconsole/spectre.console@0.47.0...0.48.0)

---
updated-dependencies:
- dependency-name: Spectre.Console
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump WeCantSpell.Hunspell from 4.0.0 to 4.1.0

Bumps [WeCantSpell.Hunspell](https://github.com/aarondandy/WeCantSpell.Hunspell) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/aarondandy/WeCantSpell.Hunspell/releases)
- [Changelog](https://github.com/aarondandy/WeCantSpell.Hunspell/blob/main/changelog.md)
- [Commits](aarondandy/WeCantSpell.Hunspell@4.0.0...4.1.0)

---
updated-dependencies:
- dependency-name: WeCantSpell.Hunspell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Task/update plugin bundling (#1678)

* attempt to wget dicom plugin

* improved looping for plugins

* update script

* add to bundle

* update build

* add charts

* update changelog

* remove bad files

* improve build

* add verbose curl

* temp diable tests

* attempt to simplify build

* readd tests

---------

Co-authored-by: James A Sutherland <[email protected]>

* Bump shogo82148/actions-setup-perl from 1.24.2 to 1.24.3

Bumps [shogo82148/actions-setup-perl](https://github.com/shogo82148/actions-setup-perl) from 1.24.2 to 1.24.3.
- [Release notes](https://github.com/shogo82148/actions-setup-perl/releases)
- [Commits](shogo82148/actions-setup-perl@v1.24.2...v1.24.3)

---
updated-dependencies:
- dependency-name: shogo82148/actions-setup-perl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* bump version

* attemt sign

* remove coveralls

* update sign

* add ls

* correct file name

* more ls

* correct linux file extention

* comment out linux

* fix typo

* update signing

* update changelog

* add ls

* remove extra sign

* add test back in

* Bump NLog from 5.2.6 to 5.2.7

Bumps [NLog](https://github.com/NLog/NLog) from 5.2.6 to 5.2.7.
- [Release notes](https://github.com/NLog/NLog/releases)
- [Changelog](https://github.com/NLog/NLog/blob/dev/CHANGELOG.md)
- [Commits](https://github.com/NLog/NLog/commits)

---
updated-dependencies:
- dependency-name: NLog
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump System.Security.Permissions from 7.0.0 to 8.0.0 (#1676)

* Bump System.Threading.AccessControl from 7.0.1 to 8.0.0 (#1675)

* Bump NUnit from 3.14.0 to 4.0.0 (#1686)

* Bump actions/setup-dotnet from 3.2.0 to 4.0.0 (#1694)

* Feature/rdmp-113 datasets (#1682)

* Dataset PR

* Bump shogo82148/actions-setup-perl from 1.24.3 to 1.25.0 (#1692)

Bump shogo82148/actions-setup-perl from 1.24.3 to 1.25.0 (#1692)

* Bump SixLabors.ImageSharp from 3.0.2 to 3.1.0 (#1693)

* Bump NUnit from 4.0.0 to 4.0.1

Bumps [NUnit](https://github.com/nunit/nunit) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/nunit/nunit/releases)
- [Changelog](https://github.com/nunit/nunit/blob/master/CHANGES.md)
- [Commits](nunit/nunit@v4.0.0...v4.0.1)

---
updated-dependencies:
- dependency-name: NUnit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump SixLabors.ImageSharp.Drawing from 2.0.1 to 2.1.0

Bumps [SixLabors.ImageSharp.Drawing](https://github.com/SixLabors/ImageSharp.Drawing) from 2.0.1 to 2.1.0.
- [Release notes](https://github.com/SixLabors/ImageSharp.Drawing/releases)
- [Commits](SixLabors/ImageSharp.Drawing@v2.0.1...v2.1.0)

---
updated-dependencies:
- dependency-name: SixLabors.ImageSharp.Drawing
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Implement first pass at platform export tool (#1672)

* Implement first pass at platform export tool

* Remove dead code, add progress report

* Syntax fix

* Note command addition in CHANGELOG

* ignore certain dqe types

---------

Co-authored-by: James A Sutherland <>
Co-authored-by: James Friel <[email protected]>

* update rdmp client xml

* Bump SixLabors.ImageSharp from 3.1.0 to 3.1.1

Bumps [SixLabors.ImageSharp](https://github.com/SixLabors/ImageSharp) from 3.1.0 to 3.1.1.
- [Release notes](https://github.com/SixLabors/ImageSharp/releases)
- [Commits](SixLabors/ImageSharp@v3.1.0...v3.1.1)

---
updated-dependencies:
- dependency-name: SixLabors.ImageSharp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump MongoDB.Driver from 2.22.0 to 2.23.0

Bumps [MongoDB.Driver](https://github.com/mongodb/mongo-csharp-driver) from 2.22.0 to 2.23.0.
- [Release notes](https://github.com/mongodb/mongo-csharp-driver/releases)
- [Commits](mongodb/mongo-csharp-driver@v2.22.0...v2.23.0)

---
updated-dependencies:
- dependency-name: MongoDB.Driver
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: jas88 <[email protected]>
Co-authored-by: James A Sutherland <[email protected]>
Co-authored-by: James A Sutherland <unknown>
Co-authored-by: Brian <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants