Apply fails silently when applying old version of existing schema

[ondrejbilcik]

Hi,

We're running into an issue when reverting changes to a schema, when terraform apply succeeds however the schema does not revert to the previous version in the registry.
We can work around this using an arbitrary version attribute, however the terraform apply should still throw an error in cases when the attribute isn't changed and thus the change isn't applied to the schema.

When TF_LOG=warn is set, the following warning is displayed when applying:

2024-06-17T17:18:19.977+0200 [WARN] Provider "provider["registry.terraform.io/confluentinc/confluent"]" produced an unexpected new value for module.schemas.confluent_schema.this["event.initiative.InitiativeUpdated"], but we are tolerating it because it is using the legacy plugin SDK.

[ConfluentSpencer]

@ondrejbilcik can you please share us some example configs and the expected outcomes? This will help us troubleshoot. CC @channingdong

[ondrejbilcik]

Sure. I set up some test configs to illustrate the issue:

1. I created the following schema:

{
  "name": "Test",
  "type": "record",
  "namespace": "test.one",
  "doc": "This is a test",
  "fields": [
    {
      "name": "test1",
      "type": "string",
      "doc": "Random string to test the schema"
    }
  ]
}

2. I updated the schema with the following:

{
	"type": "record",
	"name": "Test",
	"namespace": "test.one",
	"doc": "This is a test",
	"fields": [
		{
			"name": "test1",
			"type": "string",
			"doc": "Random string to test the schema"
		},
		{
			"name": "test2",
			"type": "string",
			"doc": "Random second string to test the schema",
			"default": null
		}
	]
}

3. I'd like to revert the change from step two, the apply succeeds

Terraform will perform the following actions:

  # module.schemas.confluent_schema.this["ondrej.test"] will be updated in-place
  ~ resource "confluent_schema" "this" {
        id                 = "1/ondrej.test/latest"
      ~ schema             = jsonencode(
          ~ {
              ~ fields    = [
                    {
                        doc  = "Random string to test the schema"
                        name = "test1"
                        type = "string"
                    },
                  - {
                      - default = null
                      - doc     = "Random second string to test the schema"
                      - name    = "test2"
                      - type    = "string"
                    },
                ]
                name      = "Test"
                # (3 unchanged attributes hidden)
            }
        )
        # (6 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.schemas.confluent_schema.this["ondrej.test"]: Modifying... [id=1/ondrej.test/latest]
module.schemas.confluent_schema.this["ondrej.test"]: Still modifying... [id=1/ondrej.test/latest, 10s elapsed]
module.schemas.confluent_schema.this["ondrej.test"]: Modifications complete after 10s [id=1/ondrej.test/latest]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

As you can see below, the schema is still on version two. I would expect either the schema to revert to the previous version, or for terraform to throw an error stating that the change wasn't successfully applied.

Let me know if you need any more data.

Cheers,
Ondrej

[Noel-Jones]

I can throw some light on this. The provider is sending the reverted schema to confluent cloud schema registry as a create schema message using the "Register Schema Under Subject" API but schema registry is saying "it already exists, here is the schema ID : 10972" (whatever your ID is for version 1). The provider is accepting this as success and there is no logic to make this the latest version. I'm not sure if the rest api actually allows this to be achieved.

[Noel-Jones]

I've checked, and you cannot do this from the clould ui either - if you click to evolve a schema and change it back to the previous version nothing is actually changed. So it really begs the question of what Terraform should do?

It could (soft) delete later versions so that the matching previous version becomes the latest. This doesn't sound like a wise course of action without due consideration of impact on applications.

It could (soft) delete the matching previous version and then recreate it so that it becomes the latest verion. This is likely to fail the backward compatibility validation (and also impact applications should be considered).

It could display a warning or error message to say that the schema is no longer the latest version. This would be my preferance since it explains why the apply is having no effect. I would lean towards a warning rather than an error since the error would stop you in your tracks. Maybe a flag to allow you to change the action "action_if_not_latest" (ignore/warn/error) or something like that.

I have this issue happening a lot with application that auto update the schema (particularly managed connectors where the ability to turn off auto schema update does not exist). The ability to change the action would at least allow me to silence Terraform (though in the case of the Managed connector schema there seems little point in trying to manage those particular schema at all).

[Dawnflash]

I am in favor of displaying a warning, preferably an error as opposed to pulling some tricks in the background. It's not that hard to manually delete old versions, devs can write runbooks for these scenarios.