chore: Support dynamic json values which Map type #3860

AgustinBettati · 2025-11-07T15:58:40Z

Description

Link to any related issue(s): CLOUDP-352319

Before

Code generation didn’t handle OpenAPI object maps with dynamic json as values, resulting in a panic during code generation.

objectOfDynamicValues:
  description: Object of dynamic values.
  type: object
  additionalProperties:
    type: object

search_index_api couldn’t express definition.mappings.fields as a map of schemaless JSON values.

After

Codegen recognizes this pattern and generates a MapValue[jsontypes.Normalized] on the terraform side.
search_index_api now exposes definition.mappings.fields enabling jsonencode(...) per field.
Add unit testing to ensure Marshal/Unmarshal support dynamic JSON (jsontypes.Normalized) across map/list/set.

Type of change:

Bug fix (non-breaking change which fixes an issue). Please, add the "bug" label to the PR.
New feature (non-breaking change which adds functionality). Please, add the "enhancement" label to the PR. A migration guide must be created or updated if the new feature will go in a major version.
Breaking change (fix or feature that would cause existing functionality to not work as expected). Please, add the "breaking change" label to the PR. A migration guide must be created or updated.
This change requires a documentation update
Documentation fix/enhancement

Required Checklist:

I have signed the MongoDB CLA
I have read the contributing guides
I have checked that this change does not generate any credentials and that they are NOT accidentally logged anywhere.
I have added tests that prove my fix is effective or that my feature works per HashiCorp requirements
I have added any necessary documentation (if appropriate)
I have run make fmt and formatted my code
If changes include deprecations or removals I have added appropriate changelog entries.
If changes include removal or addition of 3rd party GitHub actions, I updated our internal document. Reach out to the APIx Integration slack channel to get access to the internal document.

Further comments

AgustinBettati · 2025-11-07T16:05:00Z

internal/common/autogen/marshal_test.go

-		AttrDynamicJSONString  jsontypes.Normalized `tfsdk:"attr_dynamic_json_string"`
-		AttrDynamicJSONNumber  jsontypes.Normalized `tfsdk:"attr_dynamic_json_number"`
-		AttrDynamicJSONArray   jsontypes.Normalized `tfsdk:"attr_dynamic_json_array"`
+		AttrDynamicJSONObject         jsontypes.Normalized                        `tfsdk:"attr_dynamic_json_object"`


no changes to marshal/unmarshal logic needed, adding testing to ensure coverage

AgustinBettati · 2025-11-07T16:07:08Z

tools/codegen/codespec/attribute.go

+		nestedObject := NestedAttributeObject{Attributes: objectAttributes}
+		if len(nestedObject.Attributes) == 0 {
+			// objects without attributes use JSON custom type as element
+			result := s.buildRegularCollectionAttribute(name, tfModelName, computability, isSet, CustomTypeJSON)
+			return result, nil
+		}
+
+		result := s.buildNestedCollectionAttribute(name, tfModelName, fullName, computability, isSet, nestedObject)
+		return result, nil


change to list creation was purely refactoring, want to make clear when nested vs regular collection types are created, and when CustomTypeJSON is used (only within regular collection type).

Copilot

Pull Request Overview

This PR enhances the code generation tool to support OpenAPI object maps with dynamic JSON values, addressing a previous limitation that caused panics during code generation. The key change enables search_index_api to express definition.mappings.fields as a map of schemaless JSON values using MapValue[jsontypes.Normalized].

Key Changes:

Code generation now recognizes OpenAPI objects without defined properties and generates appropriate MapValue[jsontypes.Normalized] types
Error handling improved by replacing panics with proper error returns in code generation functions
Added comprehensive test coverage for marshaling/unmarshaling dynamic JSON across map/list/set types

Reviewed Changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tools/codegen/models/search_index_api.yaml	Adds `fields` map attribute to definition.mappings for search index API
tools/codegen/main.go	Updates to handle errors from code generation functions instead of ignoring them
tools/codegen/gofilegen/schema/schema_file.go	Changes GenerateGoCode to return errors instead of panicking
tools/codegen/gofilegen/resource/resource_file.go	Changes GenerateGoCode to return errors instead of panicking
tools/codegen/gofilegen/schema/schema_file_test.go	Updates tests to handle error returns from GenerateGoCode
tools/codegen/gofilegen/resource/resource_file_test.go	Updates tests to handle error returns from GenerateGoCode
tools/codegen/config.yml	Removes TODO comment and ignores for fields that are now supported
tools/codegen/codespec/testdata/config.yml	Adds test resource configuration for dynamic JSON properties
tools/codegen/codespec/testdata/api-spec.yml	Adds OpenAPI spec for testing dynamic JSON properties
tools/codegen/codespec/attribute.go	Refactors array/map attribute building to handle empty objects as dynamic JSON
tools/codegen/codespec/api_to_provider_spec_mapper_test.go	Adds test case for dynamic JSON properties conversion
internal/serviceapi/searchindexapi/resource_test.go	Adds acceptance test for mappings.fields functionality
internal/serviceapi/searchindexapi/resource_schema.go	Adds generated fields map attribute to mappings schema
internal/common/autogen/unmarshal_test.go	Expands tests to cover dynamic JSON in map/list/set types
internal/common/autogen/marshal_test.go	Expands tests to cover dynamic JSON in map/list/set types

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

lantoli · 2025-11-11T08:46:53Z

tools/codegen/codespec/api_to_provider_spec_mapper_test.go

+			}},
+		},
+	}
+	runTestCase(t, tc)


nit: i know it's not only from this PR but don't see the need to extract runTestCase logic instead of having it in each test.

yeah agree runTestCase is very lightweight, don't see harm in having it unified

lantoli · 2025-11-11T08:48:16Z

tools/codegen/codespec/attribute.go

+	if isSet {
+		attr.CustomType = NewCustomNestedSetType(nestedObjectName)
+		attr.SetNested = &SetNestedAttribute{NestedObject: nestedObject}
+	} else {
+		attr.CustomType = NewCustomNestedListType(nestedObjectName)
+		attr.ListNested = &ListNestedAttribute{NestedObject: nestedObject}
+	}


Suggested change

if isSet {

attr.CustomType = NewCustomNestedSetType(nestedObjectName)

attr.SetNested = &SetNestedAttribute{NestedObject: nestedObject}

} else {

attr.CustomType = NewCustomNestedListType(nestedObjectName)

attr.ListNested = &ListNestedAttribute{NestedObject: nestedObject}

}

attr.CustomType = NewCustomNestedListType(nestedObjectName)

attr.ListNested = &ListNestedAttribute{NestedObject: nestedObject}

if isSet {

attr.CustomType = NewCustomNestedSetType(nestedObjectName)

attr.SetNested = &SetNestedAttribute{NestedObject: nestedObject}

}

nit: don't know if it have side effect to update the values and then set again if set

I would go for the current implementation which does not allocate more than necessary (even if the compiler does optimize this, which I don't think it does).

manupedrozo · 2025-11-11T11:16:51Z

tools/codegen/gofilegen/schema/schema_file.go

 )

-func GenerateGoCode(input *codespec.Resource) []byte {
+func GenerateGoCode(input *codespec.Resource) ([]byte, error) {


AgustinBettati added 6 commits November 7, 2025 11:34

improve original error message

2a2eedf

adding failing test in api spec parsing logic

7c7d1b1

fix parsing of map type fixing leaving unit test passing

36a4d77

code generation working

f112f9d

add marshal/unmarshal unit tests

686b6fa

add acceptance test

b3f5460

AgustinBettati commented Nov 7, 2025

View reviewed changes

AgustinBettati marked this pull request as ready for review November 7, 2025 16:18

AgustinBettati requested a review from a team as a code owner November 7, 2025 16:18

Copilot AI review requested due to automatic review settings November 7, 2025 16:18

Copilot AI reviewed Nov 7, 2025

View reviewed changes

marcosuma approved these changes Nov 10, 2025

View reviewed changes

lantoli reviewed Nov 11, 2025

View reviewed changes

lantoli approved these changes Nov 11, 2025

View reviewed changes

manupedrozo approved these changes Nov 11, 2025

View reviewed changes

AgustinBettati merged commit 603f19d into master Nov 11, 2025
63 checks passed

AgustinBettati deleted the CLOUDP-352319-rebased branch November 11, 2025 11:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore: Support dynamic json values which Map type #3860

chore: Support dynamic json values which Map type #3860

Uh oh!

AgustinBettati commented Nov 7, 2025 •

edited

Loading

Uh oh!

AgustinBettati Nov 7, 2025

Uh oh!

AgustinBettati Nov 7, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

lantoli Nov 11, 2025

Uh oh!

AgustinBettati Nov 11, 2025

Uh oh!

lantoli Nov 11, 2025 •

edited

Loading

Uh oh!

manupedrozo Nov 11, 2025

Uh oh!

manupedrozo Nov 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

chore: Support dynamic json values which Map type #3860

chore: Support dynamic json values which Map type #3860

Uh oh!

Conversation

AgustinBettati commented Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Before

After

Type of change:

Required Checklist:

Further comments

Uh oh!

AgustinBettati Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

AgustinBettati Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

lantoli Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

AgustinBettati Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

lantoli Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

manupedrozo Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

manupedrozo Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

AgustinBettati commented Nov 7, 2025 •

edited

Loading

lantoli Nov 11, 2025 •

edited

Loading