Merge branch 'develop'

Documentation/CodeTutorials/Tests.md

@@ -1,4 +1,7 @@
 # RDMP Code Tests
+
+To run RDMP tests [install Sql Server Localdb](#installing-localdb) then run `dotnet test`
+
 ## Background
 Tests in RDMP are split between [Unit](#unit-tests), [User Interface](#user-interface-tests) and [Database](#database-tests) tests.
 
@@ -74,23 +77,31 @@ _Example User Interface Test_
 
 The RDMP client requires an Sql Server instance for storing platform metadata objects (`Catalogue`, `Project` etc).  its primary purpose is to query / manage SQL datasets (for linkage, extraction etc).  Database tests exist to test this functionality.
 
-The easiest way to achieve this is to install the 'SQL Server Express LocalDB' package in Visual Studio:
+The easiest way to achieve this is to install 'SQL Server Express LocalDB'.
+
+### Installing LocalDb
 
-![image](https://user-images.githubusercontent.com/31306100/199936580-dd6cbd70-625a-4a9c-8303-6cb752535b4b.png)
+You can install SQL Server Express LocalDB via the Visual Studio Installer.  Alternatively, if you are running on linux you can [install sql server express](https://learn.microsoft.com/en-us/sql/linux/quickstart-install-connect-docker?view=sql-server-ver16&pivots=cs1-bash).
+ 
+![Installing local db](./Images/Tests/InstallingLocalDb.png)
 
-If you are using LocalDB then your server will be called `(localdb)\MSSQLLocalDB`.  If you have manually installed the full version of SQL Server Express tehn it is likely to be `localhost\sqlexpress` or just `localhost`.  In both cases the user authentication will be done with your Windows account so no username or password is required.
+If you are using LocalDB then your server will be called `(localdb)\MSSQLLocalDB`.  If you have manually installed the full version of SQL Server Express then it is likely to be called `localhost\sqlexpress` or just `localhost`.
 
-Before running DatabaseTests you must create a set of RDMP platform databases for testing.  This can be done through the [RDMP command line tool](https://github.com/HicServices/RDMP/releases):
+If you are not using LocalDb you will have to update [TestDatabases.txt] to specify the correct `Server:`.  Running on linux also requires entering the container's `Username:` and `Password:` options.  On windows integrated security will automatically handle authentication based on your user account.
 
-`rdmp.exe install localhost\sqlexpress TEST_ -D`
+The first time you run `dotnet test` RDMP will create the testing databases listed in [TestDatabases.txt](../../Tests.Common/TestDatabases.txt) provided that:
 
-Or you can use the client application at startup:
+- The server exists and can be connected to
+- The Prefix listed contains 'TEST' (case insensitive).
 
-![ReOrdering](Images/CreatePlatformDatabases.png) 
+If this process fails or you need to manually recreate them then this can be done with the [RDMP command line](./RdmpCommandLine.md).  You can either run it from the repository or fetch the binary from [Github Releases](https://github.com/HicServices/RDMP/releases).
 
-If you need to change the server name or database prefix from the above example then update ".\Tests.Common\DatabaseTests.txt" to match.
+```
+cd ./Tools/rdmp/
+dotnet run -- install "(localdb)\MSSQLLocalDB" TEST_ -D
+```
 
-If you have a testing environment with an Oracle and\or MySql server instance then you can enable running these tests too by entering the connection strings into `DatabaseTests.txt`.  If you do not define a connection string then these tests will be marked `Inconclusive` when run.
+If you have a testing environment with an Oracle, Postgres and\or MySql server instance then you can enable running these tests too by entering the connection strings into [TestDatabases.txt].  If you do not define a connection string then these tests will be marked `Inconclusive` when run.
 
 __WARNING__: DatabaseTests will delete the contents of the TEST_ databases before each test is run and some will create temporary databases/tables during runtime, therefore it is important that you do not use a production server for integration testing
 
@@ -189,3 +200,4 @@ public class MyTests : DatabaseTests
 [Catalogue]: ./Glossary.md#Catalogue
 
 [Project]: ./Glossary.md#Project
+[TestDatabases.txt]: ../../Tests.Common/TestDatabases.txt

Documentation/CodeTutorials/UserManual.md

@@ -80,7 +80,9 @@ You can install the client software via the [installer link on the Github site](
 It is possible to run RDMP as a standalone tool without an Sql Server instance using a [File System Backend](./YamlRepository.md) but this approach is recommended only for single user or standalone systems where RDMP performs a specific activity in isolation (e.g. data load, cohort creation etc).
 
 ## System Requirements
-You will need an Sql Server instance (unless using a [file system backend](./YamlRepository.md)).  If you do not already have one, you can use the Express edition for free which is available from microsoft.com (https://www.microsoft.com/en-us/sql-server/sql-server-editions-express ).  Alternatively you can use [LocalDb](https://learn.microsoft.com/en-us/sql/database-engine/configure-windows/sql-server-express-localdb?view=sql-server-ver16) which is a Sql Server development tool installable with Visual Studio that allows for instances to be started when needed and shutdown automatically.
+You will need an Sql Server instance (unless using a [file system backend](./YamlRepository.md)).  Sql Server Express is free.  Alternatively you can install LocalDb from the Visual Studio Installer:
+
+![Platform database setup](Images/Tests/InstallingLocalDb.png)
 
 You will also need to have your actual dataset data in a relational database.  RDMP supports accessing research data repositories stored in MySql, Sql Server and Oracle databases but stores it's own metadata (platform databases) in Sql Server only.
 

README.md

@@ -88,24 +88,12 @@ cd bin/Debug/net6.0
 
 ### Tests
 
-To run tests you will need an instance of SQL Server.  These instructions use localdb which is included in [visual studio](https://docs.microsoft.com/en-us/sql/database-engine/configure-windows/sql-server-express-localdb?view=sql-server-ver15).
-
-If using a docker container or alternate sql server instance then substitute your host name in place of `(localdb)\MSSQLLocalDB`
+To run tests you will need an instance of SQL Server.  These instructions use LocalDb which is an optional installable package in Visual Studio.  See [Tests.md](./Documentation/CodeTutorials/Tests.md) for more information on setting up LocalDb, testing on linux or using an alternate server as well as information on the types of tests included and how to write new ones.
 
 ```
-dotnet build
-./Tools/rdmp/bin/Debug/net6.0/rdmp.exe install "(localdb)\MSSQLLocalDB" TEST_ -d
-
-echo "ServerName: (localdb)\MSSQLLocalDB" > ./Tests.Common/TestDatabases.txt
-echo "Prefix: TEST_" >> ./Tests.Common/TestDatabases.txt
-
-dotnet build
-
 dotnet test ./scripts/run-all-tests.proj -c Release -p:BuildInParallel=false
 ```
 
-For a more indepth guide to CI testing see [How to set up your test environment in Tests.md](Documentation/CodeTutorials/Tests.md).
-
 ## Contributing
 
 We welcome all contributions including:

Tests.Common/DatabaseTests.cs

@@ -58,6 +58,7 @@ public class DatabaseTests
     {
         protected readonly IRDMPPlatformRepositoryServiceLocator RepositoryLocator;
         protected static TestDatabasesSettings TestDatabaseSettings;
+        private static bool HaveTriedCreatingTestDatabases;
 
         public ICatalogueRepository CatalogueRepository
         {
@@ -166,7 +167,10 @@ public DatabaseTests()
                     $"Expecting Unit Test Catalogue To Be At Server={cataRepo.DiscoveredServer.Name} Database={cataRepo.DiscoveredServer.GetCurrentDatabase()}");
 
                 if(!cataRepo.DiscoveredServer.Exists())
-                    Assert.Inconclusive("Catalogue database does not exist, run 'rdmp.exe install ...' to create it (Ensure that servername and prefix in TestDatabases.txt match those you provide to CreateDatabases.exe e.g. 'rdmp.exe install localhost\\sqlexpress TEST_')");
+                {
+                    DealWithMissingTestDatabases(opts,cataRepo);
+                }
+
             }
 
 
@@ -225,6 +229,29 @@ public DatabaseTests()
                 _discoveredPostgresServer = new DiscoveredServer(TestDatabaseSettings.PostgreSql, DatabaseType.PostgreSql);
         }
 
+        private void DealWithMissingTestDatabases(PlatformDatabaseCreationOptions opts, TableRepository cataRepo)
+        {
+            var mainDb = cataRepo.DiscoveredServer.ExpectDatabase("master");
+
+            if (HaveTriedCreatingTestDatabases || !mainDb.Server.Exists())
+            {
+                Assert.Inconclusive("Test database server does not exist.  You must install SQL Server LocalDb or Sql Server Express to run DatabaseTests. Or update TestDatabases.txt to point to your existing server.");
+            }
+
+            // if user is trying to connect to a test database
+            // and that server exists (but TEST_ databases don't)
+            if (opts.Prefix.Contains("TEST",StringComparison.InvariantCultureIgnoreCase))
+            {
+                // then create them
+                TestContext.Out.WriteLine($"Creating TEST databases on {mainDb.Server} using prefix {opts.Prefix}");
+
+                var creator = new PlatformDatabaseCreation();
+                creator.CreatePlatformDatabases(opts);
+            }
+
+            HaveTriedCreatingTestDatabases = true;
+        }
+
         private IDataExportRepository GetFreshYamlRepository()
         {
             var dir = new DirectoryInfo(Path.Combine(TestContext.CurrentContext.WorkDirectory, "Repo"));