SqlDataReader returns stale data #980
Comments
|
Hi @dawust Could you share more details with code snippets about how do you open and end transactions? The error "Internal .NET Framework Data Provider error 60." refers to "UnknownTransactionFailure" which is triggered when driver cannot initialize delegated transaction here: But as you said cast issues happen before that error, so something has gone bad already on your connection/transaction before this is raised. To figure that out, is it possible to collect below information for same failure?
|
|
Thanks for your reply. We've got our own in-house developed ORM, thus luckily all of our transactions follow the same pattern. public static class TransactionScopeUtils
{
public static TransactionScope BeginScope(bool requireSeparateTransaction = false)
{
return new TransactionScope(
requireSeparateTransaction ? TransactionScopeOption.RequiresNew : TransactionScopeOption.Required,
new TransactionOptions { IsolationLevel = IsolationLevel.ReadCommitted },
TransactionScopeAsyncFlowOption.Enabled);
}
}
/// Sample read usage
using (TransactionScopeUtils.BeginScope(requireSeparateTransaction: true))
{
// Read data following this pattern
using (var sqlConnection = new SqlConnection(connectionString))
{
using (var sqlCommand = new SqlCommand(sqlText, sqlConnection))
{
sqlConnection.Open();
using (var reader = sqlCommand.ExecuteReader(CommandBehavior.SequentialAccess))
{
while (reader.Read())
{
var column1 = reader.GetInt32(0);
var column2 = reader.GetDateTime(1);
var column3 = reader.GetString(2);
...
}
}
}
}
}All of our write operations make use of a TransactionScope according to the following pattern: using (var transactionScope = new TransactionScope(
scopeOption: TransactionScopeOption.Required,
transactionOptions: new TransactionOptions { Timeout = transactionTimeout, IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted, },
asyncFlowOption: TransactionScopeAsyncFlowOption.Enabled))
{
var retryCount = 3;
while (retryCount > 0)
{
try
{
using (var sqlConnection = new SqlConnection(connectionString))
{
// Do all our writing which involves reading multiple tables, writing multiple tables (sometimes using SqlBulkCopy)
}
}
catch (SqlException exception) when (sqlException.Number == 1205 && retryCount > 1)
{
// a sql exception that is a deadlock
Thread.Sleep(new Random().Next(100));
retryCount--;
}
}
transactionScope.Complete();
}I hope the code fragments aren't simplified too much. I'm really grateful for your support. |
|
I managed to create a repro for both Exceptions: Hope this helps! |
|
Thanks for the repro @dawust ! |
|
Adding some notes for readers:
More investigations to find root cause are in progress. |
|
Hi @cheenamalhotra, |
|
Yes this issue is on a different path and not related to #659. |
|
@DavoudEshtehari Thanks for the fix! I have rolled out the fix in one of our secondary systems. On this particular system we observe the issue once every two or three days. I'll give you an update on the situation by next week. |
|
It seems that the fix did the trick, no more exceptions :) |
|
Hi @dawust, thank you for the quick check. This change will apply as a hot fix to the stable versions as well as 2.x. |
Describe the bug
On rare occasions SqlDataReader returns stale data from a previous query.
We read the columns of a SELECT statement sequentially using ExecuteReader and the corresponding Get operations, eg.
Eventhough the Get operations match the data types of the given SELECT statement in the SqlCommand, sometimes System.InvalidCastException is thrown.
When investating the DataTable of reader.GetSchemaTable() it was obvious that the result belonged to a preceding unrelated SELECT statement. We experience this behaviour a few dozen times a day over millions of queries. It seems that the SqlConnection is corrupted, then is put back in the ConnectionPool and subsequent queries using the same SqlConnection are affected as well until the connection is doomed eventually.
This behaviour is often accompanied by a subsequent request failing with the following InvalidOperationException exception "Internal .NET Framework Data Provider error 60."
To reproduce
Unfortunately I don't have a repro. It seems to happen sporadically and is not related to heavy load.
Expected behavior
SqlDataReader only returns data corresponding to the given SqlCommand.
Further technical details
Microsoft.Data.SqlClient version: 2.1.2
.NET target: .NET 5.0
SQL Server version: SQL Server 2019
Operating system: Both Windows and Linux (Docker container mcr.microsoft.com/dotnet/aspnet:5.0)
Additional context
Both async and sync operations are affected. Some of our queries are wrapped in a TransactionScope.
To mitigate the issue, I exposed the Abort method of the SqlConnection using reflection, wrapped everything in a try/catch block and doom the connection manually after InvalidOperation or InvalidCast exceptions. By doing so subsequent queries are less prone to be affected.
Thank you in advance for any insights!
The text was updated successfully, but these errors were encountered: