Remove resource state attributes that are no longer in the schema

[jbardin]

While removal of attributes can be handled by providers through the
UpgradeResourceState call, data sources may need to be evaluated before
reading, and they have no upgrade path in the provider protocol.
Strip out extra attributes during state decoding when they are no longer
present in the schema, and there is no schema upgrade pending. If there
is a pending schema upgrade, let the provider handle it since it may want to
check for the removed attribute values.

Another option not explored here is declaring that providers must migrate
any attributes pending removal before they actually removed, which would
allow the possibility of making the decoder less strict and always strip out
unknown attributes (similar to encoding/json). Since we can
handle the case fully within core, it seems reasonable to let the decoder be
as strict as possible, though it might be useful to implement the functionality
upstream for other projects using zclconf.

The validate command should work only with the configuration, but when
validate was run at the start of a plan or apply command the state was
inserted in preparation for the next walk. This could lead to errors
when the resource schemas had changes and the state could not be
upgraded or decoded.

Fixes #25752

[codecov]

Codecov Report

Merging #25779 into master will increase coverage by 0.00%.
The diff coverage is 80.00%.

Impacted Files	Coverage Δ
terraform/eval_state_upgrade.go	63.15% <79.59%> (+17.50%)	⬆️
backend/local/backend_local.go	37.91% <83.33%> (+1.02%)	⬆️
checkpoint.go	4.76% <0.00%> (-28.58%)	⬇️
terraform/node_resource_plan.go	91.80% <0.00%> (-1.64%)	⬇️
terraform/evaluate.go	53.46% <0.00%> (-0.45%)	⬇️
internal/providercache/dir.go	75.86% <0.00%> (+5.17%)	⬆️
command/ui_input.go	66.23% <0.00%> (+5.19%)	⬆️

[pselle]

Nice update

[ghost]

Is there any ETA on .13.1 release? Gasping for my TF stacks to work again 👍

[renan]

Yes, please release a new version!

[ChristophShyper]

So now, if I import some resource that was created in other way and possibly modified, I have no clue which attributes were set to which values? If I destroy it and recreate it with Terraform then this resource can have different attributes' values.

That is not a bugfix, that is a breaking change and should be loudly advertised.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.