Skip to content

Commit

Permalink
Merge in #9859 to v8/feature/nucache-perf (#10152)
Browse files Browse the repository at this point in the history
* load only once

* Bump version to 8.6.8

* Initial rework of Lock dictionaries

* [Issue 5277-146] accessibility - Close 'X' icon next to language drop… (#9264)

* [Issue 5277-146] accessibility - Close 'X' icon next to language drop down is identified as "link" - screen reader

* add new loacalization key

* Fix issue with SqlMainDomLock that cannot use implicit lock timeouts … (#9973)

* Fix issue with SqlMainDomLock that cannot use implicit lock timeouts … (#9973)

(cherry picked from commit da5351d)

* Adjust unit tests and apply fixes to scope

* Add more unit tests, showing current issue

* Counting Umbraco.ModelsBuilder and ModelsBuilder.Umbraco namespaces as external providers

* Fix dead lock with TypeLoader

* Fix errors shown in unit tests

* Throw error if all scopes hasn't been disposed

* Clean

* Fixes and Updates for DB Scope and Ambient Context leaks (#9953)

* Adds some scope tests (ported back from netcore) and provides a much better error message, ensure execution context is not flowed to child tasks that shouldn't leak any current ambient context

* updates comment

* Ensure SqlMainDomLock suppresses execution context too

* Since we're awaiting a task in a library method, ConfigureAwait(false)

* missing null check

Co-authored-by: Elitsa Marinovska <[email protected]>

* Adds additional error checking and reporting to MainDom/SqlMainDomLock (#9954)

Co-authored-by: Elitsa Marinovska <[email protected]>

* Add copy logic to Media Picker (#9957)

* Add copy logic to Media Picker

* Add action for copy all

* Fix for selectable media item

* Wrap calls to map in scopes

* Autocomplete scopes

* Remove unnecessary aria-hidden attribute from <umb-icon>

* Remove scope from method that calls another method that has a scope

* Fixes #9993 - Cannot save empty image in Grid

* Clean

* Revert "The Value() method for IPublishedContent was not working with the defaultValue parameter" (#9989)

* Use a hashset to keep track of acquired locks

This simplifies disposing/checking for locks greatly.

* Add images in grid - fixes 9982 (#9987)

Co-authored-by: Sebastiaan Janssen <[email protected]>

* Only create the dicts and hashset when a lock is requested

* Clean

* Adds a config for configuring the access rules on the content dashboard - by default it granted for all user groups

* Adds additional params indicating whether user is admin

* Add images in grid - fixes 9982 (#9987)

Co-authored-by: Sebastiaan Janssen <[email protected]>
(cherry picked from commit e201977)

* Bump version to 8.12.2

* #9964 Removed unneeded check for HttpContext

* Fix for #9950 - HttpsCheck will now retry using the login background image if inital request returns 301/302. Excessvie Headers check will now check the root url instead of the backoffice

* Merge pull request #9994 from umbraco/v8/bugfix/9993

Fixes #9993 - Cannot save empty image in Grid

(cherry picked from commit 0ecc933)

* Apply suggestions from review

* Fixes #9983 - Getting kicked, if document type has a Umbraco.UserPicker property (#10002)

* Fixes #9983

Temporary fix for this issue. using the entityservice like before.

* Needed to remove the call to usersResource here as well for displaying the picked items

* Don't need usersResource for now

* Fixes #9983 - Getting kicked, if document type has a Umbraco.UserPicker property (#10002)

* Fixes #9983

Temporary fix for this issue. using the entityservice like before.

* Needed to remove the call to usersResource here as well for displaying the picked items

* Don't need usersResource for now

(cherry picked from commit 45de0a1)

* 8539: Allow alias in image cropper (#9266)

Co-authored-by: Owain Williams <[email protected]>

* Wrap dumping dictionaries in a method.

* Create method for generating log message

And remove forgotten comments.

* Fix swedish translation for somethingElse.

* Copy member type (#10020)

* Add copy dialog for member type

* Implement copy action for member type

* Create specific localization for content type, media type and member type

* Handle "foldersonly" querystring

* Add button type attribute

* Add a few missing changes of anchor to button element

* Null check on scope and options to ensure backward compatibility

* Improve performance, readability and handling of FollowInternalRedirects (#9889)

* Improve performance, readability and handling of FollowInternalRedirects

* Logger didn't like string param

Passing string param to _logger.Debug<PublishedRouter, int> wasn't happy. Changed to pass existing internalRedirectAsInt variable.

Co-authored-by: Nathan Woulfe <[email protected]>

* Update casing of listview layout name

* 9097 add contextual password helper (#9256)

* update back-office forms

* Display tip on reset password page as well

* add directive for password tip

* integrate directove in login screen

* forgot the ng-keyup :-)

* adapt tooltip directive to potential different Members and Users password settings

* remove watcher

Co-authored-by: Nathan Woulfe <[email protected]>

* Unbind listener

Listening for splitViewRequest was only unbound if the split view editor was opened. Not cleaning up the listener caused a memory leak when changing between nodes as the spit view editor was detached but not garbage-collected

* Replace icon in date picker with umb-icon component (#10040)

* Replace icon in date picker with <umb-icon> component

* Adjust height of clear button

* Update cypress and fix tests

* Listview config icons (#10036)

* Update icons to use <umb-icon> component

* Simplify markup and use disabled button

* Use move cursor style on sortable handle

* Add class for action column

* Update setting auto focus

* Increase font size of umb-panel-header-icon

* Anchor noopener (#10009)

* Set rel="noopener" for anchors with target="_blank"

* Reverted unwanted changes to Default.cshtml

* Align 'Add language' test to netcore

* Add new cypress tests

* Add indentation

* Getting rid of the config file and implementing an appSetting instead

* Implementation for IContentDashboardSettings

* Cleanup

* bool.Try

* Taking AllowContentDashboardAccessToAllUsers prop from GlobalSettings to ContentDashboardSettings and saving AccessRulesFromConfig into a backing field

* fix support for non run states

* Handling multiple values per field in Examine Management

* Add Root<T> and Breadcrumbs extension methods for IPublishedContent (#9033)

* Fix usage of obsolete CreatorName and WriterName properties

* Add generic Root extension method

* Add Breadcrumbs extension methods

* Orders member type grouping of members alphabetically, matching the listing of member types.

* Revert updating deprecated WriterName/CreatorName refs

Changing the properties to use the extensions is a good thing (given the props are deprecated), but causes issues deep in tests. I'm reverting that change to fix the tests, and all refs to the deprecated properties should be updated in one sweep, to deal with any other test issues that might crop up.

* Handle Invalid format for Upgrade check

* Fixes tabbing-mode remains active after closing modal #9790 (#10074)

* Allow to pass in boolean to preventEnterSubmit directive (#8639)

* Pass in value to preventEnterSubmit directive

* Set enabled similar to preventDefault and preventEnterSubmit directives

* Update prevent enter submit value

* Init value from controller

* Use a different default input id prefix for umb-search-filter

* Fix typo

* Check for truthly value

* Revert "Set enabled similar to preventDefault and preventEnterSubmit directives"

This reverts commit 536ce85.

* None pointer events when clicking icon

* Use color variable

* Fixes tabbing-mode remains active after closing modal #9790 (#10074)

(cherry picked from commit c881fa9)

* Null check on scope and options to ensure backward compatibility

(cherry picked from commit fe8cd23)

* Fix validation of step size in integer/numeric field

* 9962: Use $allowedEditors instead of allowed (#10086)

* 9962: Use $allowedEditors instead of allowed

* 9962: Remove redundant statement

* fixes #10021 adds ng-form and val-form-manager to the documentation

* Improved accessibility of link picker (#10099)

* Added support for screeen reader alerts on the embed so that assitive technology knows when a url retrieve has been succesfull.
Added labels for the controls
Preview reload only triggered if the values for height and width change

* Added control ids for the link picker

* Add French translation

* Accessibility: Alerts the user how many results have been returned on a tree search (#10100)

* Added support for screeen reader alerts on the embed so that assitive technology knows when a url retrieve has been succesfull.
Added labels for the controls
Preview reload only triggered if the values for height and width change

* Tree search details the number of search items returned

* Add French translations

* Updated LightInject to v6.4.0

* Remove HtmlSanitizer once more - see #9803

* Also make sure NuGet installs the correct version of the CodePages dependency

* Bump version to 8.13 RC

* Fixed copy preserving sort order (#10091)

* Revert "Updated LightInject to v6.4.0"

This reverts commit fc77252.

* Revert "Add copy logic to Media Picker (#9957)"

This reverts commit f7c032a.

* Reintroduce old constructor to make non-breaking

* Update cypress test to make macros in the grid work again

* Attributes could be multiple items, test specifically if `Directory` is an attribute

* Accessibility: Adding label fors and control ids for the macro picker (#10101)

* Added support for screeen reader alerts on the embed so that assitive technology knows when a url retrieve has been succesfull.
Added labels for the controls
Preview reload only triggered if the values for height and width change

* Added support for label fors for the macro picker and also gave the ,acro search box a title

* Now displays a count of the matching macros returned. Please note the language file amends shared with #10100

* Removed src-only class for the display of the count of messages

* Updating typo

* Removed top-margin from switcher icon

* Allow KeepAlive controller Ping method to be requested by non local requests (#10126)

* Allow KeepAlive controller Ping method to be requested by non local requests and accept head requests

* removed unused references

* fix csproj

* fix merge

* btree serializer optimizations

* array pool and nametable optimizations

Co-authored-by: Mole <[email protected]>
Co-authored-by: Sebastiaan Janssen <[email protected]>
Co-authored-by: Justin Shearer <[email protected]>
Co-authored-by: Bjarke Berg <[email protected]>
Co-authored-by: Callum Whyte <[email protected]>
Co-authored-by: Shannon <[email protected]>
Co-authored-by: Elitsa Marinovska <[email protected]>
Co-authored-by: patrickdemooij9 <[email protected]>
Co-authored-by: Bjarne Fyrstenborg <[email protected]>
Co-authored-by: Michael Latouche <[email protected]>
Co-authored-by: Nathan Woulfe <[email protected]>
Co-authored-by: Markus Johansson <[email protected]>
Co-authored-by: Jeavon Leopold <[email protected]>
Co-authored-by: Benjamin Carleski <[email protected]>
Co-authored-by: Owain Williams <[email protected]>
Co-authored-by: Jesper Löfgren <[email protected]>
Co-authored-by: Martin Bentancour <[email protected]>
Co-authored-by: Ronald Barendse <[email protected]>
Co-authored-by: Andy Butland <[email protected]>
Co-authored-by: BeardinaSuit <[email protected]>
Co-authored-by: Mads Rasmussen <[email protected]>
Co-authored-by: Rachel Breeze <[email protected]>
Co-authored-by: Dave de Moel <[email protected]>
Co-authored-by: ric <[email protected]>
Co-authored-by: Carole Rennie Logan <[email protected]>
Co-authored-by: Dennis Öhman <[email protected]>
  • Loading branch information
1 parent 98d8971 commit 7704d02
Show file tree
Hide file tree
Showing 16 changed files with 232 additions and 38 deletions.
88 changes: 59 additions & 29 deletions src/Umbraco.Core/Sync/DatabaseServerMessenger.cs
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ namespace Umbraco.Core.Sync
// but only processes instructions coming from remote servers,
// thus ensuring that instructions run only once
//
public class DatabaseServerMessenger : ServerMessengerBase
public class DatabaseServerMessenger : ServerMessengerBase, ISyncBootStateAccessor
{
private readonly IRuntimeState _runtime;
private readonly ManualResetEvent _syncIdle;
Expand Down Expand Up @@ -172,15 +172,39 @@ private void Initialize(IUmbracoDatabase database)
lock (_locko)
{
if (_released) return;
var coldboot = IsColdBoot(database);

var coldboot = false;
if (_lastId < 0) // never synced before
if (coldboot)
{
// we haven't synced - in this case we aren't going to sync the whole thing, we will assume this is a new
// server and it will need to rebuild it's own caches, eg Lucene or the xml cache file.
Logger.Warn<DatabaseServerMessenger>("No last synced Id found, this generally means this is a new server/install."
+ " The server will build its caches and indexes, and then adjust its last synced Id to the latest found in"
+ " the database and maintain cache updates based on that Id.");
// go get the last id in the db and store it
// note: do it BEFORE initializing otherwise some instructions might get lost
// when doing it before, some instructions might run twice - not an issue
var maxId = database.ExecuteScalar<int>("SELECT MAX(id) FROM umbracoCacheInstruction");

//if there is a max currently, or if we've never synced
if (maxId > 0 || _lastId < 0)
SaveLastSynced(maxId);

// execute initializing callbacks
if (Options.InitializingCallbacks != null)
foreach (var callback in Options.InitializingCallbacks)
callback();
}

_initialized = true;
}
}

private bool IsColdBoot(IUmbracoDatabase database)
{
var coldboot = false;
if (_lastId < 0) // never synced before
{
// we haven't synced - in this case we aren't going to sync the whole thing, we will assume this is a new
// server and it will need to rebuild it's own caches, eg Lucene or the xml cache file.
Logger.Warn<DatabaseServerMessenger>("No last synced Id found, this generally means this is a new server/install."
+ " The server will build its caches and indexes, and then adjust its last synced Id to the latest found in"
+ " the database and maintain cache updates based on that Id.");

coldboot = true;
}
Expand All @@ -198,29 +222,11 @@ private void Initialize(IUmbracoDatabase database)
+ " to the latest found in the database and maintain cache updates based on that Id.",
count, Options.MaxProcessingInstructionCount);

coldboot = true;
}
}

if (coldboot)
{
// go get the last id in the db and store it
// note: do it BEFORE initializing otherwise some instructions might get lost
// when doing it before, some instructions might run twice - not an issue
var maxId = database.ExecuteScalar<int>("SELECT MAX(id) FROM umbracoCacheInstruction");

//if there is a max currently, or if we've never synced
if (maxId > 0 || _lastId < 0)
SaveLastSynced(maxId);

// execute initializing callbacks
if (Options.InitializingCallbacks != null)
foreach (var callback in Options.InitializingCallbacks)
callback();
coldboot = true;
}

_initialized = true;
}

return coldboot;
}

/// <summary>
Expand Down Expand Up @@ -548,6 +554,30 @@ private string GetDistCacheFilePath(IGlobalSettings globalSettings)

#endregion

public SyncBootState GetSyncBootState()
{
try
{
ReadLastSynced(); // get _lastId
using (var scope = ScopeProvider.CreateScope())
{
EnsureInstructions(scope.Database);
bool isColdBoot = IsColdBoot(scope.Database);

if (isColdBoot)
{
return SyncBootState.ColdBoot;
}
return SyncBootState.HasSyncState;
}
}
catch(Exception ex)
{
Logger.Warn<DatabaseServerMessenger>("Error determining Sync Boot State", ex);
return SyncBootState.Unknown;
}
}

#region Notify refreshers

private static ICacheRefresher GetRefresher(Guid id)
Expand Down
20 changes: 20 additions & 0 deletions src/Umbraco.Core/Sync/ISyncBootStateAccessor.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Umbraco.Core.Sync
{
/// <summary>
/// Retrieve the state of the sync service
/// </summary>
public interface ISyncBootStateAccessor
{
/// <summary>
/// Get the boot state
/// </summary>
/// <returns></returns>
SyncBootState GetSyncBootState();
}
}
19 changes: 19 additions & 0 deletions src/Umbraco.Core/Sync/NonRuntimeLevelBootStateAccessor.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Umbraco.Core.Sync
{
/// <summary>
/// Boot state implementation for when umbraco is not in the run state
/// </summary>
public class NonRuntimeLevelBootStateAccessor : ISyncBootStateAccessor
{
public SyncBootState GetSyncBootState()
{
return SyncBootState.Unknown;
}
}
}
24 changes: 24 additions & 0 deletions src/Umbraco.Core/Sync/SyncBootState.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Umbraco.Core.Sync
{
public enum SyncBootState
{
/// <summary>
/// Unknown state. Treat as HasSyncState
/// </summary>
Unknown = 0,
/// <summary>
/// Cold boot. No Sync state
/// </summary>
ColdBoot = 1,
/// <summary>
/// Warm boot. Sync state present
/// </summary>
HasSyncState = 2
}
}
3 changes: 3 additions & 0 deletions src/Umbraco.Core/Umbraco.Core.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,9 @@
<Compile Include="Migrations\Upgrade\V_8_7_0\MissingDictionaryIndex.cs" />
<Compile Include="Services\IInstallationService.cs" />
<Compile Include="Services\IUpgradeService.cs" />
<Compile Include="Sync\ISyncBootStateAccessor.cs" />
<Compile Include="Sync\NonRuntimeLevelBootStateAccessor.cs" />
<Compile Include="Sync\SyncBootState.cs" />
<Compile Include="SystemLock.cs" />
<Compile Include="Attempt.cs" />
<Compile Include="AttemptOfTResult.cs" />
Expand Down
2 changes: 2 additions & 0 deletions src/Umbraco.Tests/PublishedContent/NuCacheChildrenTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
using Umbraco.Core.Services;
using Umbraco.Core.Services.Changes;
using Umbraco.Core.Strings;
using Umbraco.Core.Sync;
using Umbraco.Tests.TestHelpers;
using Umbraco.Tests.Testing.Objects;
using Umbraco.Tests.Testing.Objects.Accessors;
Expand Down Expand Up @@ -158,6 +159,7 @@ private void Init(Func<IEnumerable<ContentNodeKit>> kits)
Mock.Of<IEntityXmlSerializer>(),
Mock.Of<IPublishedModelFactory>(),
new UrlSegmentProviderCollection(new[] { new DefaultUrlSegmentProvider() }),
new TestSyncBootStateAccessor(SyncBootState.HasSyncState),
_contentNestedDataSerializerFactory);

// invariant is the current default
Expand Down
2 changes: 2 additions & 0 deletions src/Umbraco.Tests/PublishedContent/NuCacheTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
using Umbraco.Core.Services;
using Umbraco.Core.Services.Changes;
using Umbraco.Core.Strings;
using Umbraco.Core.Sync;
using Umbraco.Tests.TestHelpers;
using Umbraco.Tests.Testing.Objects;
using Umbraco.Tests.Testing.Objects.Accessors;
Expand Down Expand Up @@ -204,6 +205,7 @@ private void Init()
Mock.Of<IEntityXmlSerializer>(),
Mock.Of<IPublishedModelFactory>(),
new UrlSegmentProviderCollection(new[] { new DefaultUrlSegmentProvider() }),
new TestSyncBootStateAccessor(SyncBootState.HasSyncState),
_contentNestedDataSerializerFactory);

// invariant is the current default
Expand Down
1 change: 1 addition & 0 deletions src/Umbraco.Tests/Scoping/ScopedNuCacheTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ protected override IPublishedSnapshotService CreatePublishedSnapshotService()
Factory.GetInstance<IEntityXmlSerializer>(),
Mock.Of<IPublishedModelFactory>(),
new UrlSegmentProviderCollection(new[] { new DefaultUrlSegmentProvider() }),
new TestSyncBootStateAccessor(SyncBootState.HasSyncState),
nestedContentDataSerializerFactory);
}

Expand Down
2 changes: 2 additions & 0 deletions src/Umbraco.Tests/Services/ContentTypeServiceVariantsTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
using Umbraco.Core.Services;
using Umbraco.Core.Strings;
using Umbraco.Core.Sync;
using Umbraco.Tests.TestHelpers;
using Umbraco.Tests.TestHelpers.Entities;
using Umbraco.Tests.Testing;
using Umbraco.Web.PublishedCache;
Expand Down Expand Up @@ -73,6 +74,7 @@ protected override IPublishedSnapshotService CreatePublishedSnapshotService()
Factory.GetInstance<IEntityXmlSerializer>(),
Mock.Of<IPublishedModelFactory>(),
new UrlSegmentProviderCollection(new[] { new DefaultUrlSegmentProvider() }),
new TestSyncBootStateAccessor(SyncBootState.HasSyncState),
nestedContentDataSerializerFactory);
}

Expand Down
23 changes: 23 additions & 0 deletions src/Umbraco.Tests/TestHelpers/TestSyncBootStateAccessor.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Umbraco.Core.Sync;

namespace Umbraco.Tests.TestHelpers
{
class TestSyncBootStateAccessor : ISyncBootStateAccessor
{
private readonly SyncBootState _syncBootState;

public TestSyncBootStateAccessor(SyncBootState syncBootState)
{
_syncBootState = syncBootState;
}
public SyncBootState GetSyncBootState()
{
return _syncBootState;
}
}
}
1 change: 1 addition & 0 deletions src/Umbraco.Tests/Umbraco.Tests.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@
<Compile Include="Services\RedirectUrlServiceTests.cs" />
<Compile Include="Templates\HtmlLocalLinkParserTests.cs" />
<Compile Include="TestHelpers\RandomIdRamDirectory.cs" />
<Compile Include="TestHelpers\TestSyncBootStateAccessor.cs" />
<Compile Include="Testing\Objects\TestDataSource.cs" />
<Compile Include="Published\PublishedSnapshotTestObjects.cs" />
<Compile Include="Published\ModelTypeTests.cs" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ public override void Compose(Composition composition)

composition.SetDatabaseServerMessengerOptions(GetDefaultOptions);
composition.SetServerMessenger<BatchedDatabaseServerMessenger>();
composition.Register<ISyncBootStateAccessor>(factory=> factory.GetInstance<IServerMessenger>() as BatchedDatabaseServerMessenger, Lifetime.Singleton);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ internal class DictionaryOfPropertyDataSerializer : SerializerBase, ISerializer<
{
public IDictionary<string, PropertyData[]> ReadFrom(Stream stream)
{
var dict = new Dictionary<string, PropertyData[]>(StringComparer.InvariantCultureIgnoreCase);

// read properties count
var pcount = PrimitiveSerializer.Int32.ReadFrom(stream);

var dict = new Dictionary<string, PropertyData[]>(pcount,StringComparer.InvariantCultureIgnoreCase);
// read each property
for (var i = 0; i < pcount; i++)
{
Expand All @@ -28,13 +28,13 @@ public IDictionary<string, PropertyData[]> ReadFrom(Stream stream)
var vcount = PrimitiveSerializer.Int32.ReadFrom(stream);

// create pdata and add to the dictionary
var pdatas = new List<PropertyData>();
var pdatas = new PropertyData[vcount];

// for each value, read and add to pdata
for (var j = 0; j < vcount; j++)
{
var pdata = new PropertyData();
pdatas.Add(pdata);
pdatas[j] =pdata;

// everything that can be null is read/written as object
// even though - culture and segment should never be null here, as 'null' represents
Expand All @@ -46,7 +46,7 @@ public IDictionary<string, PropertyData[]> ReadFrom(Stream stream)
pdata.Value = ReadObject(stream);
}

dict[key] = pdatas.ToArray();
dict[key] = pdatas;
}
return dict;
}
Expand Down
Loading

0 comments on commit 7704d02

Please sign in to comment.