CLI-3467: Downstream Terraform Resources of confluent_schema.<name>.schema Throw invalid new value for... (2.20.0)

[matthew-coudert-cko]

Summary

The Confluent Terraform provider (version 2.20.0) modifies the whitespace to "normalize" Protobuf schemas string during the terraform apply process.

Since this provider was made using the provider plugin v1, this change to a non-computed value in the schema is allowed. If any "downstream" resources try to use this data as an input (via confluent_schema.schema.schema) then the downstream provider crashes as a "required" input does not match the value during plan time.

I chose the local provider to provide a reproducible example (below), but for other use cases this can cause real problems.

Steps to Reproduce

1. Create main.tf:

   terraform {
     required_providers {
       confluent = {
         source  = "confluentinc/confluent"
         version = "2.20.0"
       }
       local = {
         source  = "hashicorp/local"
         version = "2.5.2"
       }
     }
   }
   
   provider "confluent" {
     schema_registry_id            = "schema-registry"
     schema_registry_rest_endpoint = "[https://schemas.cko-qa.ckotech.co](https://schemas.cko-qa.ckotech.co)"
     schema_registry_api_key       = " "
     schema_registry_api_secret    = " "
   }
   
   resource "confluent_schema" "schema" {
     subject_name = "test.reproduce"
     schema       = file("reproduce.proto")
     format       = "PROTOBUF"
   }
   
   resource "local_file" "test" {
     content  = confluent_schema.schema.schema // Trying to utilise this data in a downstream resource
     filename = "output_schema.txt"
   }

2. Create reproduce.proto:

   syntax = "proto3";
   package test;
   
   message Reproduce {
     string name =
         1;
   }

3. Run terraform init:

   terraform init

4. Run terraform apply:

   terraform apply

Expected Behavior

The local_file.test resource should successfully create the output_schema.txt file with the exact content of reproduce.proto.

Actual Behavior

The terraform apply command fails with the following error:

Error: Provider produced inconsistent final plan

When expanding the plan for local_file.test to include new values learned so far during apply, provider "registry.terraform.io/hashicorp/local" produced an invalid new value for .content: was cty.StringVal("syntax = "proto3";\npackage test;\n\nmessage Reproduce {\n  string name =\n      1;\n}"), but now cty.StringVal("syntax = "proto3";\npackage test;\n\nmessage Reproduce {\n  string name = 1;\n}\n").

This is a bug in the provider, which should be reported in the provider's own issue tracker.

The Confluent provider alters the whitespace in the schema string, specifically removing the extra spaces around the field number 1 in the reproduce.proto message. This difference in whitespace causes the local provider to detect a change in the content attribute, leading to the "inconsistent final plan" error.

Terraform and Provider Versions

• Terraform: 1.11.1
• Confluent Provider: 2.20.0
• Local Provider: 2.5.2

I was able to reproduce this bug on 2.19.0 and 2.9.0 as well.

Mitigations

• If you ensure that your proto file is exactly normalized than this succeeds (as confluent doesn't make any changes to it). I don't think this is something we can expect users to do (especially as the error isn't clear about what's going on).
• If you instead consume the schema directly from the file (rather than from the schema resource this bug is avoidable. This can lead to out of sync issues though if you change one resources file but not the other!

   resource "local_file" "test" {
      content  = file("reproduce.proto") // Don't consume directly from the confluent_schema resource. 
      filename = "output_schema.txt"
    }

[linouk23]

@matthew-coudert-cko thanks for creating this issue!

It could be a very tricky issue to resolve. Out of curiosity, did you try using the confluent_schema data source instead?

data "confluent_schema" "purchase-v1" {
  subject_name = "proto-purchase-value"
  schema_identifier = 10001
}

[matthew-coudert-cko]

We can use the confluent_schema data source. But if they are part of the same terraform stack we can't show anything downstream of the confluent_schema during the planning stage.

To give you some context for our use case we (in the same stack):

• Deploy a protobuf confluent_schema to CSR.
• Deploy a Kafka Topic to MSK.
  Based on that protobuf schema we generate and deploy the "table version" of that schema in:
• Hive
• BigQuery
• Snowflake
• Iceberg
• Etc...

What we currently do is show the "plan" of the table that its going to generate from the given schema (so they can make changes before they deploy and we can raise breaking changes (invalid field names etc...) as part of the plan).

If we use the data source instead we can't show the planned "effect" of their changes (as it won't be deployed yet). Hopefully that makes sense!

[sajjadlateef]

Hi @matthew-coudert-cko , I wanted to share that this issue will require some time to investigate. If this is blocking you in some way, please reach out to Confluent Support and file a ticket.

[matthew-coudert-cko]

Thanks for letting me know! Let me know if I can be helpful in anyway.