Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tentacle reports a System.IO.DirectoryNotFoundException because it could not find a part of the path #660

Open
1 task done
sburmanoctopus opened this issue Nov 8, 2023 · 0 comments
Open
1 task done

Tentacle reports a System.IO.DirectoryNotFoundException because it could not find a part of the path #660

sburmanoctopus opened this issue Nov 8, 2023 · 0 comments
Labels
team/fire-and-motion

Comments

@sburmanoctopus
Copy link
Contributor

Team

  • I've assigned a team label to this issue

What happened?

Tentacle sometimes reports a System.IO.DirectoryNotFoundException because it could not find a part of the path.

Reproduction

This was noticed while monitoring for RPC retry errors. It would only happen rarely, but it was noticed.

Error and Stacktrace

Server exception: 
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.IO.DirectoryNotFoundException: Could not find a part of the path.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.File.InternalMove(String sourceFileName, String destFileName, Boolean checkHost)
   at Halibut.Transport.Protocol.TemporaryFileStream.SaveTo(String filePath)
   at System.Collections.Generic.List`1.ForEach(Action`1 action)
   at Octopus.Tentacle.Scripts.ScriptWorkspaceFactory.PrepareWorkspace(ScriptTicket ticket, String scriptBody, Dictionary`2 scripts, ScriptIsolationLevel isolationLevel, TimeSpan scriptMutexAcquireTimeout, String scriptMutexName, String[] scriptArguments, List`1 files) in C:\\BuildAgent\\work\\639265b01610d682\\source\\Octopus.Tentacle\\Scripts\\ScriptWorkspaceFactory.cs:line 70
   at Octopus.Tentacle.Services.Scripts.ScriptServiceV2.StartScript(StartScriptCommandV2 command) in C:\\BuildAgent\\work\\639265b01610d682\\source\\Octopus.Tentacle\\Services\\Scripts\\ScriptServiceV2.cs:line 63
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Halibut.ServiceModel.ServiceInvoker.Invoke(RequestMessage requestMessage)
   at Halibut.Transport.Protocol.MessageExchangeProtocol.InvokeAndWrapAnyExceptions(RequestMessage request, Func`2 incomingRequestProcessor)
   at Halibut.ServiceModel.HalibutProxyWithAsync.ThrowExceptionFromReceivedError(ServerError error, ILog logger)
   at Halibut.ServiceModel.HalibutProxyWithAsync.MakeRpcCall(MethodInfo asyncMethod, Object[] args)
   at Halibut.ServiceModel.HalibutProxyWithAsync.InvokeAsyncT[T](MethodInfo asyncMethod, Object[] args)
   at System.Reflection.AsyncDispatchProxyGenerator.InvokeAsync[T](Object[] args)
   at Octopus.Tentacle.Client.Decorators.HalibutExceptionAsyncScriptServiceV2Decorator.StartScriptAsync(StartScriptCommandV2 command, HalibutProxyRequestOptions halibutProxyRequestOptions)
   at Octopus.Tentacle.Client.Scripts.ScriptExecutionOrchestrator.<>c__DisplayClass14_1.<<ExecuteScript>b__2>d.MoveNext()
--- End of stack trace from previous location ---
   at Octopus.Tentacle.Client.Utils.AsyncHalibutFeatureExtensionMethods.WhenEnabled[T](AsyncHalibutFeatureWithResult`1 asyncHalibutFeatureWithResult, Func`1 action)
   at Octopus.Tentacle.Client.Scripts.ScriptExecutionOrchestrator.<>c__DisplayClass14_0.<<ExecuteScript>g__StartScriptAction|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Octopus.Tentacle.Client.Execution.RpcCallExecutor.<>c__DisplayClass7_1`1.<<ExecuteWithRetries>b__3>d.MoveNext()
--- End of stack trace from previous location ---
   at Octopus.Tentacle.Client.Execution.RpcCallExecutor.<>c__DisplayClass7_0`1.<<ExecuteWithRetries>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Octopus.Tentacle.Client.Retries.RpcCallRetryHandler.<>c__DisplayClass11_0`1.<<ExecuteWithRetries>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Octopus.Tentacle.Client.Retries.RpcCallRetryHandler.<>c__DisplayClass10_0`1.<<ExecuteWithRetries>g__ExecuteAction|2>d.MoveNext()
--- End of stack trace from previous location ---
   at Polly.Retry.AsyncRetryEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates`1 shouldRetryResultPredicates, Func`5 onRetryAsync, Int32 permittedRetryCount, IEnumerable`1 sleepDurationsEnumerable, Func`4 sleepDurationProvider, Boolean continueOnCapturedContext)
   at Polly.AsyncPolicy.ExecuteAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
   at Octopus.Tentacle.Client.Retries.RpcCallRetryHandler.ExecuteWithRetries[T](Func`2 action, OnRetyAction onRetryAction, OnTimeoutAction onTimeoutAction, CancellationToken cancellationToken)
   at Octopus.Tentacle.Client.Retries.RpcCallRetryHandler.ExecuteWithRetries[T](Func`2 action, OnRetyAction onRetryAction, OnTimeoutAction onTimeoutAction, Boolean abandonActionOnCancellation, TimeSpan abandonAfter, CancellationToken cancellationToken)
   at Octopus.Tentacle.Client.Execution.RpcCallExecutor.ExecuteWithRetries[T](RpcCall rpcCall, Func`2 action, ILog logger, Boolean abandonActionOnCancellation, ClientOperationMetricsBuilder clientOperationMetricsBuilder, CancellationToken cancellationToken)
   at Octopus.Tentacle.Client.Scripts.ScriptExecutionOrchestrator.ExecuteScript(CancellationToken scriptExecutionCancellationToken)
   at Octopus.Tentacle.Client.TentacleClient.ExecuteScript(StartScriptCommandV2 startScriptCommand, Action`1 onScriptStatusResponseReceived, Func`2 onScriptCompleted, ILog logger, CancellationToken scriptExecutionCancellationToken)
   at Octopus.Server.Orchestration.Targets.Tentacles.TentacleRemoteEndpointFacadeV2.ExecuteCommand(StartScriptCommand startScriptCommand, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/Targets/Tentacles/TentacleRemoteEndpointFacadeV2.cs:line 71
   at Octopus.Server.Orchestration.Targets.Common.RemoteEndpointFacadeCancellationTokenDecorator.ExecuteCommand(StartScriptCommand command, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/Targets/Common/RemoteEndpointFacadeCancellationTokenDecorator.cs:line 36
   at Octopus.Server.Orchestration.Targets.Tentacles.Observability.ErrorLoggingRemoteEndpointFacadeDecorator.ExecuteCommand(StartScriptCommand command, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/Targets/Tentacles/Observability/ErrorLoggingRemoteEndpointFacadeDecorator.cs:line 75
   at Octopus.Server.Orchestration.ServerTasks.Deploy.ActionExecution.Immediate.ExecutionTargets.TentacleExecutionTarget.Execute(ScriptCollection bootstrapperScripts, IReadOnlyList`1 bootstrapperArguments, IReadOnlyList`1 files, Nullable`1 forceIsolationLevel, Boolean raw, ITaskLog taskLog, String isolationMutexName, CancellationToken cancellationToken, Nullable`1 isolationMutexTimeout) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/ActionExecution/Immediate/ExecutionTargets/TentacleExecutionTarget.cs:line 70
   at Octopus.Server.Orchestration.ServerTasks.Deploy.ActionExecution.Immediate.ImmediateExecutor.RunOnTarget(ScriptCollection finalBootstrapperScripts, String platform, IBootstrapperScriptGenerator bootstrapperScriptGenerator, IReadOnlyList`1 bootstrapperArguments, IReadOnlyList`1 allFiles, Nullable`1 isolation, Nullable`1 isolationMutexTimeout, String isolationMutexName, CalamariFlavour flavour, IReadOnlyList`1 deploymentTools, ExecutionTargetDetails executionTargetDetails, CalamariPlatformConstraint calamariPlatformConstraint, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/ActionExecution/Immediate/ImmediateExecutor.cs:line 292
   at Octopus.Server.Orchestration.ServerTasks.Deploy.ActionExecution.Immediate.ImmediateExecutor.ExecuteCalamari(CalamariFlavour calamariFlavour, String calamariCommand, IReadOnlyList`1 calamariArguments, IReadOnlyList`1 files, IReadOnlyList`1 deploymentTools, VariableCollection extraVariables, TargetManifest targetManifest, CalamariPlatformConstraint calamariPlatformConstraint, Nullable`1 isolationMutexTimeout, String isolationMutexName, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/ActionExecution/Immediate/ImmediateExecutor.cs:line 224
   at Octopus.Server.Orchestration.ServerTasks.Deploy.ActionExecution.CommandBuilders.CalamariCommandBuilder.Execute(ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/ActionExecution/CommandBuilders/CalamariCommandBuilder.cs:line 162
   at Octopus.Server.Orchestration.ServerTasks.Deploy.Steps.Acquire.PackageAcquisitionExecutionHandlers.<>c__DisplayClass7_0.<<ApplyDelta>b__0>d.MoveNext() in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/Steps/Acquire/PackageAcquisitionExecutionHandlers.cs:line 136
--- End of stack trace from previous location ---
   at Octopus.Server.Orchestration.ServerTasks.Deploy.ActionDispatch.AdHocActionDispatcher.InvokeActionHandler(Machine target, ActionHandlerInvocation actionHandler, ActionAndTargetScopedVariables actionAndTargetScopedVariables, IExecutor executor, TargetManifest targetManifest, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/ActionDispatch/AdHocActionDispatcher.cs:line 238
   at Octopus.Server.Orchestration.ServerTasks.Deploy.ActionDispatch.AdHocActionDispatcher.ExecuteOnDeploymentTarget(DeploymentTarget deploymentTarget, ActionHandlerInvocation actionHandler, TargetManifest targetManifest, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/ActionDispatch/AdHocActionDispatcher.cs:line 195
   at Octopus.Server.Orchestration.ServerTasks.Deploy.ActionDispatch.AdHocActionDispatcher.Dispatch(Machine machine, ActionHandlerInvocation actionHandler, ITaskLog taskLog, CancellationToken cancellationToken, VariableCollection variables) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/ActionDispatch/AdHocActionDispatcher.cs:line 80
   at Octopus.Server.Orchestration.ServerTasks.Deploy.Steps.Acquire.PackageAcquisitionExecutionHandlers.Invoke(ActionHandlerInvocation invocation, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/Steps/Acquire/PackageAcquisitionExecutionHandlers.cs:line 150
   at Octopus.Server.Orchestration.ServerTasks.Deploy.Steps.Acquire.PackageAcquisitionExecutionHandlers.ApplyDelta(StoredPackage nearestLocalPackage, Stream packageDeltaFile, String fullPathOnRemoteMachine, String deltaFileName, String newFileName, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/Steps/Acquire/PackageAcquisitionExecutionHandlers.cs:line 135
   at Octopus.Server.Orchestration.Targets.Common.PackageStagingMediator.UploadAndApplyDelta(StoredPackage packageToPush, StoredPackage nearestLocalPackage, String fullPathOnRemoteMachine, PackageAcquisitionExecutionHandlers execs, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/Targets/Common/PackageStagingMediator.cs:line 324
   at Octopus.Server.Orchestration.Targets.Common.PackageStagingMediator.TryPushPackageAsDelta(IPackageFeedWithLocalCache packageFeed, StoredPackage packageToPush, IEnumerable`1 existingRemotePackages, PackageAcquisitionExecutionHandlers execs, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/Targets/Common/PackageStagingMediator.cs:line 262
   at Octopus.Server.Orchestration.Targets.Common.PackageStagingMediator.PushPackageWithDeltaCompression(StoredPackage packageToPush, PackageIdentity packageIdentity, IDeploymentPackageDownloader deploymentPackageDownloader, PackageAcquisitionExecutionHandlers execs, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/Targets/Common/PackageStagingMediator.cs:line 200
   at Octopus.Server.Orchestration.Targets.Common.PackageStagingMediator.PushPackage(DependencyIdentity identity, StoredPackage packageToPush, IDeploymentPackageDownloader deploymentPackageDownloader, DeltaCompressionOptions deltaCompressionOptions, PackageCacheOptions packageCacheOptions, PackageAcquisitionExecutionHandlers execs, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/Targets/Common/PackageStagingMediator.cs:line 145
   at Octopus.Server.Orchestration.Targets.Common.PackageStagingMediator.DownloadPackageAndPushToAgent(DependencyIdentity dependencyIdentity, IDeploymentPackageDownloader deploymentPackageDownloader, DeltaCompressionOptions deltaCompressionOptions, PackageCacheOptions packageCacheOptions, PackageAcquisitionExecutionHandlers execs, ITaskLog stepTaskLog, ITaskLog deploymentTargetTaskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/Targets/Common/PackageStagingMediator.cs:line 99
   at Octopus.Server.Orchestration.ServerTasks.Deploy.Steps.Acquire.AcquireMachinePackageTask.StagePackage(PackageAcquisitionExecutionHandlers packageAcquisitionHandlers, ITaskLog stepTaskLog, ITaskLog deploymentTargetTaskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/Steps/Acquire/AcquireMachinePackageTask.cs:line 227
   at Octopus.Server.Orchestration.ServerTasks.Deploy.Steps.Acquire.AcquireMachinePackageTask.<>c__DisplayClass27_0.<<Acquire>b__0>d.MoveNext() in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/Steps/Acquire/AcquireMachinePackageTask.cs:line 134
--- End of stack trace from previous location ---
   at Octopus.Core.Threading.SingleResultProcess`1.Perform(Func`2 workerTask, Action logAction, CancellationToken cancellationToken) in ./source/Octopus.Core/Threading/SingleResultProcess.cs:line 45
   at Octopus.Core.Threading.SingleWorkerDoesWorkCoordinator`1.DoWork(String key, Func`2 workerTask, Action logAction, CancellationToken cancellationToken) in ./source/Octopus.Core/Threading/SingleWorkerDoesWorkCoordinator.cs:line 43
   at Octopus.Server.Orchestration.ServerTasks.Deploy.Steps.Acquire.AcquireMachinePackageTask.Acquire(AcquiredPackageMap acquiredPackageMap, ITaskLog stepTaskLog, ITaskLog deploymentTargetTaskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/Steps/Acquire/AcquireMachinePackageTask.cs:line 132
   at Octopus.Server.Orchestration.ServerTasks.Deploy.Steps.Acquire.AcquireMachinePackageTask.Acquire(AcquiredPackageMap acquiredPackageMap, ITaskLog stepTaskLog, ITaskLog deploymentTargetTaskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/Steps/Acquire/AcquireMachinePackageTask.cs:line 156
   at Octopus.Server.Orchestration.ServerTasks.Deploy.Steps.Acquire.AcquireMachinePackageTask.Acquire(AcquiredPackageMap acquiredPackageMap, ITaskLog stepTaskLog, ITaskLog deploymentTargetTaskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/Steps/Acquire/AcquireMachinePackageTask.cs:line 183
   at Octopus.Server.Orchestration.ServerTasks.Deploy.PlannedStepControllers.AcquirePackagesStepController.<>c__DisplayClass4_3`3.<<Execute>b__5>d.MoveNext() in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/PlannedStepControllers/AcquirePackagesStepController.cs:line 78
--- End of stack trace from previous location ---
   at Octopus.Server.Orchestration.ServerTasks.Deploy.Guidance.ExecuteWithoutGuidance(Func`2 callback, String actionName, Boolean actionIsRequiredToRun, ITaskLog taskLog, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/Guidance.cs:line 149
   at Octopus.Server.Orchestration.ServerTasks.Deploy.Guidance.Execute(Func`2 callback, String actionName, Boolean actionIsRequiredToRun, ITaskLog taskLog, Action callbackOnExclude, CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/Guidance.cs:line 79
   at Octopus.Server.Orchestration.ServerTasks.Deploy.PlannedStepControllers.AcquirePackagesStepController.<>c__DisplayClass4_3`3.<<Execute>b__4>d.MoveNext() in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/PlannedStepControllers/AcquirePackagesStepController.cs:line 85
--- End of stack trace from previous location ---
   at Octopus.Server.Orchestration.ServerTasks.Deploy.TransientErrorDetectionExecutor.Execute(Func`2 action, ExecutionPlan plan, ITaskLog taskLog, CancellationToken cancellationToken, DeploymentTarget deploymentTarget) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/TransientErrorDetectionExecutor.cs:line 50
   at Octopus.Server.Orchestration.ServerTasks.Deploy.TransientErrorDetectionExecutor.Execute(Func`2 action, ExecutionPlan plan, ITaskLog taskLog, CancellationToken cancellationToken, DeploymentTarget deploymentTarget) in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/TransientErrorDetectionExecutor.cs:line 50
   at Octopus.Server.Orchestration.ServerTasks.Deploy.PlannedStepControllers.AcquirePackagesStepController.<>c__DisplayClass4_2`3.<<Execute>b__3>d.MoveNext() in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/PlannedStepControllers/AcquirePackagesStepController.cs:line 89
--- End of stack trace from previous location ---
   at Octopus.Server.Infrastructure.Orchestration.UnitsOfWork.UnitOfWorkExecutor.<>c__DisplayClass6_0`3.<<Execute>b__0>d.MoveNext() in ./source/Octopus.Server/Infrastructure/Orchestration/UnitsOfWork/UnitOfWorkExecutor.cs:line 99
--- End of stack trace from previous location ---
   at Octopus.Core.Infrastructure.UnitsOfWork.UnitOfWorkExtensionMethods.Do(IUnitOfWork unitOfWork, Func`1 action, CancellationToken cancellationToken, String name) in ./source/Octopus.Core/Infrastructure/UnitsOfWork/UnitOfWorkExtensionMethods.cs:line 58
   at Octopus.Core.Infrastructure.UnitsOfWork.UnitOfWorkExtensionMethods.Do(IUnitOfWork unitOfWork, Func`1 action, CancellationToken cancellationToken, String name) in ./source/Octopus.Core/Infrastructure/UnitsOfWork/UnitOfWorkExtensionMethods.cs:line 58
   at Octopus.Server.Infrastructure.Orchestration.UnitsOfWork.UnitOfWorkExecutor.Execute[T1,T2,T3](Func`5 action, CancellationToken cancellationToken, String name) in ./source/Octopus.Server/Infrastructure/Orchestration/UnitsOfWork/UnitOfWorkExecutor.cs:line 102
   at Octopus.Server.Infrastructure.Orchestration.UnitsOfWork.UnitOfWorkExecutor.Execute[T1,T2,T3](Func`5 action, CancellationToken cancellationToken, String name) in ./source/Octopus.Server/Infrastructure/Orchestration/UnitsOfWork/UnitOfWorkExecutor.cs:line 102
   at Octopus.Server.Orchestration.ServerTasks.Deploy.PlannedStepControllers.AcquirePackagesStepController.<>c__DisplayClass4_1`3.<<Execute>b__2>d.MoveNext() in ./source/Octopus.Server/Orchestration/ServerTasks/Deploy/PlannedStepControllers/AcquirePackagesStepController.cs:line 92
--- End of stack trace from previous location ---
   at Octopus.Server.Orchestration.ServerTasks.ParallelWorkOrderAsync`1.<>c__DisplayClass7_1.<<Execute>b__3>d.MoveNext() in ./source/Octopus.Server/Orchestration/ServerTasks/ParallelWorkOrderAsync.cs:line 61
--- End of stack trace from previous location ---
   at Octopus.Server.Orchestration.ServerTasks.OctoThreadClosureAsync`1.Execute(CancellationToken cancellationToken) in ./source/Octopus.Server/Orchestration/ServerTasks/OctoThreadClosureAsync.cs:line 61

More Information

No response

Workaround

No response
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
team/fire-and-motion
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@sburmanoctopus