typebox <--> json

[sroussey]

I have a usecase where some things are defined directly in typebox. but other times I have something from an api that i want to convert to typebox. Or I want to use typebox in my code, but need to save a type definition to a database so i need to save out just the json version and then bring it back.

For example:

const Example = Type.Object({
  name: Type.String({ default: "John Doe"),
  age: Type.Number({ default: 30 }),
},{
  title: "Example",
  description: "An example object"
});

And I want to send this to another computer via API so I need to just send the JSON and then on the other side I need to take that JSON and make it into a Type.Object.

[sinclairzx81]

@sroussey Hi,

There is some new tech in V1 in support of this, Dynamic Validation and Structured Inference.

Dynamic Validation

V1 has a (mostly) 2020-12 compliant validator, so it's possible to pass raw JSON Schema to Value.Check(...) or Compile(...) and TypeBox will just treat it as a normal type. You no longer need to convert the schematic to TSchema before using it with validation (as was the case for V0). The only limitation is that dynamically loaded schematics cannot be used with the Type.* compositor (so you can't make a dynamically loaded schema Partial(...) yet), but I will be working on some tools for this in later revisions.

Structured Inference

Inline with the native JSON Schema validation support, V1 also includes Structured Inference which is able to infer raw JSON Schema. Structured Inference can be useful if loading schematics from remote sources and saving as .ts files. If TypeScript can see the full schema, then TypeBox can infer the schema via Static<T>.

Reference

The following is a quick example Reference

import Value from 'typebox/value'
import Type from 'typebox'

// -----------------------------------------------------------
// Dynamic Validation
// -----------------------------------------------------------

const T = Type.Object({
  x: Type.Number(),
  y: Type.Number(),
  z: Type.Number(),
})

// write schema to disk

Deno.writeTextFileSync('./schema.json', JSON.stringify(T))

// read schema from disk

const S = JSON.parse(Deno.readTextFileSync('./schema.json'))

Value.Check(S, { x: 1, y: 2, z: 3 })              // true

Value.Check(S, { x: 1, y: 2, z: 'not-a-number' }) // false

// -----------------------------------------------------------
// Structured Inference
// -----------------------------------------------------------

// can infer if schema structure is observable to TypeScript

type T = Type.Static<{                         // type T = {
  type: 'object',                              //   x: number;
  required: ['x', 'y', 'z'],                   //   y: number;
  properties: {                                //   z: number;
    x: { type: 'number' },                     // }
    y: { type: 'number' },
    z: { type: 'number' }
  }
}>

Dynamic Schema (Future)

This doesn't exist yet, but being able to load schematics over HTTP, then have loaded schematics computed and mapped by the Type.* compositor is on the roadmap. The main issue is that Type.* only operates on a subset of the JSON Schema specification, but where schematics loaded from elsewhere may contain structures, references, dynamic references, hyperlinks, and other "things" Type.* can't reason about.

The main issue is knowing what to do with keywords like oneOf (ExclusiveOr) as well as conditional logic (if/then/else), things that often appear in schematics but can't be neatly expressed in Type.* (because it's limited to the semantics of TypeScript). I am considering a new project just to experiment with remapping the entire JSON Schema specification into Type.*, with whatever new structures that fall out of that work being integrated into a TBV2. That's the plan anyway.

---

Hope this helps
S

[sroussey]

Hi!

At this point, I am not worried about arbitrary schemas. Just schemas that I am creating and persisting in a database. No generation of TS required, though some day that might be nice.

In fact, my requirements are pretty confined in that it is always a TObject though some properties can be optional and some can be optionally an array (string or string[]). The actual value types are pretty restricted.

[sroussey]

For toJSON= (Example)=>JSON.stringify(Example) is it. For fromJSON = (json)=> json as TObject does not cut it. I could parse the json and create the TObject and the same for each property, but it just seems like there must be something builtin. If not, I can just do that.

[sinclairzx81]

@sroussey Hiya,

---

For fromJSON = (json)=> json as TObject does not cut it

If loading serialized JSON Schema from disk, over http or a database, TypeScript won't be able to see the loaded JSON Schema structures. At most, TypeScript will either see the schema as Uint8Array | string (serialized) or unknown (deserialized) and therefore it's not possible to generate a static type level TObject dynamically because there is not enough information to safely derive a static TObject from a unknown type at runtime.

This said, there was some code written for V0 that handles reconstructing a TObject at runtime only ... this would need porting to V1.

https://github.com/sinclairzx81/typebox/blob/master/example/prototypes/from-schema.ts#L29-L251

However, with all this said, TypeBox can validate raw JSON Schema in V1 so you shouldn't need to transform to TObject at all. This was a big feature enhancement in V1 where serialized schematics could be loaded over a network and used for validation, as well as TypeBox being able to consume schematics produced by other systems / languages.

---

I could parse the json and create the TObject and the same for each property, but it just seems like there must be something builtin. If not, I can just do that.

If loading schemas dynamically, I would recommend treating the schematics as entirely opaque.

const [schema, value] = await db.readEntry(...)

if(Value.Check(schema, value)) {
   // here you don't know what the schema is, or what value is, but you 
   // do know that the value matches the schema. Any subsequent logic 
   // should treat the `value` as unknown (opaque), or if there is a requirement
   // to access properties of value, they should be explicitly checked 
   // on (top of the schema check) to ensure TypeScript can statically
   // reason about the value.
}

The above is generally going to be fine, but you need to give up statically checked access on value. There isn't anything TypeBox can provide to make dynamically loaded schematics type safe (as everything is happening at runtime), but if you really need to generate TSchema types from database types, code generation is likely the best option to get some level of type safety.

The following project is quite good

https://github.com/xddq/schema2typebox

Again hope this helps
S

[sroussey]

Hi!

I am not always checking values (though that as well). I sometimes need to compatibility between schemas. Like the intersection where two schemas are compat. For example, two TObject where each as a property named input, and one is string and the other is string | string[] (I am writing in TS syntax just so i don't have to write out the long json version, but it is the json version i need to compare).

I tend to work in typebox to define most the static stuff (the stuff with code attached), but there are user created structures saved in a database where this is not the case. I don't need code generation (or TS types) for this part. I just like keeping everything the same "form".

But I could take a different path and just work with json schema directly (maybe use types via json-schema package). I imagine structuredClone(Example) would give me the plain object to cast. I just would prefer to stick with one system of types.

BTW: for reference, the thing I am finally getting around to is to implement this:

Each of those nodes is code and the input and output are defined using Type.Object. The graph itself has an input and output though... that is calculated, saved to disk, and then reloaded later. And the compatibility of types (not values, not yet) are for connecting nodes to each other. Hope the concrete example makes more sense.