Proposal: Support for Custom Components in Chatkit

[MatthieuBonnardot]

Hi team,

First off, I really appreciate the amazing work you’re doing with Chatkit — it’s been great to build on top of it.

I currently have a branch ready across chatkit, chatkit-python, and chatkit-react that enables support for custom components. The core idea is to introduce a new widget called CustomWidget, which accepts the necessary props and JSON configuration. On the frontend, these would then be hydrated with user-defined custom components.

In theory, everything looks good with my current changes. I’ve also implemented a hook to list available custom components and automatically hydrate them in chat. However, since the Chatkit CDN code is minified, I don’t have a straightforward way to fully test this end-to-end.

Would it still be useful for me to open these PRs so the team can review the approach and direction? Or is this not aligned with OpenAI’s current vision for widgets?

I genuinely believe that enabling custom components could significantly improve flexibility, enhance customization, and drive greater adoption of Chatkit.

Thanks for your time and the great work you’re doing!

==============================

Proposed Implementation

I have a working implementation across all three repos that I'm happy to contribute.

1. Core Widget Type (@openai/chatkit)

// packages/chatkit/types/widgets.d.ts

export type WidgetComponent =
  | TextComponent
  | Title
  | Caption
  // ... existing types
  | CustomComponent;  // New type

export type CustomComponent = {
  type: 'Custom';
  key?: string;
  id?: string;
  componentType: string;  // Identifier for the React component
  props?: Record<string, unknown>;  // Props to pass to the component
} & BlockProps;

2. Widget Options Extension

// packages/chatkit/types/index.d.ts

export type WidgetsOption = {
  onAction?: (...) => Promise<void>;
  
  // New callback for custom component rendering
  onRenderCustom?: (
    component: Widgets.CustomComponent,
    container: HTMLElement,
  ) => void | Promise<void>;
};

3. React Hook (@openai/chatkit-react)

// packages/chatkit-react/src/useCustomWidgets.tsx

export type CustomComponentRegistry = Record<string, React.ComponentType<any>>;

export function useCustomWidgets(components: CustomComponentRegistry) {
  const rootsRef = React.useRef<Map<HTMLElement, ReactDOM.Root>>(new Map());

  const onRenderCustom = React.useCallback(
    (component: Widgets.CustomComponent, container: HTMLElement) => {
      const Component = components[component.componentType];

      if (!Component) {
        console.warn(
          `Custom component "${component.componentType}" not found in registry.`
        );
        return;
      }

      // Clean up existing root if it exists
      const existingRoot = rootsRef.current.get(container);
      if (existingRoot) {
        existingRoot.unmount();
      }

      // Create new root and render
      const root = ReactDOM.createRoot(container);
      rootsRef.current.set(container, root);

      root.render(
        <React.StrictMode>
          <Component {...(component.props || {})} />
        </React.StrictMode>
      );
    },
    [components]
  );

  // Cleanup on unmount
  React.useEffect(() => {
    const roots = rootsRef.current;
    return () => {
      for (const root of roots.values()) {
        root.unmount();
      }
      roots.clear();
    };
  }, []);

  return { widgets: { onRenderCustom } };
}

Usage Example

Frontend

import { useChatKit, useCustomWidgets, ChatKit } from '@openai/chatkit-react';

// Define custom component
function ProductCard({ name, price, imageUrl }: ProductCardProps) {
  return (
    <div className="product-card">
      <img src={imageUrl} alt={name} />
      <h3>{name}</h3>
      <p>${price.toFixed(2)}</p>
      <button>Add to Cart</button>
    </div>
  );
}

function App() {
  // Register custom components
  const customWidgets = useCustomWidgets({
    'product-card': ProductCard,
    'user-profile': UserProfile,
    'chart': ChartWidget,
  });

  const chatKit = useChatKit({
    api: { url: '/api/chatkit', domainKey: 'key' },
    widgets: customWidgets.widgets,
  });

  return <ChatKit control={chatKit.control} />;
}

Backend (Python SDK)

from chatkit import Card
from chatkit.widgets import CustomComponent

widget = Card(
    children=[
        CustomComponent(
            component_type="product-card",
            props={
                "name": "Wireless Headphones",
                "price": 149.99,
                "imageUrl": "https://example.com/image.jpg"
            }
        )
    ]
)

Backend (JSON)

{
  "type": "Card",
  "children": [
    {
      "type": "Custom",
      "componentType": "product-card",
      "props": {
        "name": "Wireless Headphones",
        "price": 149.99,
        "imageUrl": "https://example.com/image.jpg"
      }
    }
  ]
}

[tylersmith-openai]

Hey @MatthieuBonnardot!

First off, I really appreciate the amazing work you’re doing with Chatkit — it’s been great to build on top of it.

Thank you, really appreciate that!

I genuinely believe that enabling custom components could significantly improve flexibility, enhance customization, and drive greater adoption of Chatkit.

Thank you for sharing this proposal! There's a few technical challenges in this proposed design, specifically around getting your code and styles into the encapsulated ChatKit iframe. But, I'm curious what the primary goals/unlocks are for you in enabling this custom widget syntax:

• Improving developer experience/authoring?
• Enabling styling treatments you can't achieve in the current system?
• Enabling components that are impossible due to missing low-level components from Widgets?
• (something else?)

In the example you shared above, ProductCard looks like it would be well supported by existing widget components - akin to the Purchase Complete example.

Our goal with widgets is to provide all of the low-level components necessary to build interesting/valuable components from those primitives. The primary escape hatch we're considering is adding an Iframe component, which would enable you to render your own hosted code as the source of a widget (e.g., very complex charting/data viz, custom maps integration, etc.).

I'd love to see any specific examples you can share of where you're unable to get to the outcomes you're hoping to with widgets, to see what we can unlock there. We're rapidly expanding available widget components, as well as their capabilities - we're excited to share more about that soon.

[wangxj03]

Enabling components that are impossible due to missing low-level components from Widgets?

That is the motivation for my case.

I have been playing around with both the chatkit and apps sdk. The latter seems to allow developers to build arbitrary widgets with low-level components. For example, I cannot find an easy way to implement the carousel of place cards using the chatkit.

[MatthieuBonnardot]

Hi @tylersmith-openai

Thanks for the quick reply! The main challenge I’m facing with the current system is the limited charting capabilities. I work extensively with pricing data, and users often want to “chat with their data.” Right now, the Recharts library feels restrictive compared to something like ECharts, which offers much richer visualization options.

The example I shared earlier was meant to illustrate how my implementation would work — I should have included a more relevant use case...

Another thought is that within a frontend design system, it would be great to bring in familiar components already used across the app or website to maintain consistency in the chat experience.

Most importantly, having the option to use custom components could enable early adoption while the charting library continues to evolve. I also believe an iframe-based solution could unlock a lot of value quickly.

[tylersmith-openai]

I have been playing around with both the chatkit and apps sdk. The latter seems to allow developers to build arbitrary widgets with low-level components.

Yep, correct. We currently have different approaches, but are aiming at aligning support for both ergonomics in both products in the future. No exact timeline here.

For example, I cannot find an easy way to implement the carousel of place cards using the chat

Thank you for this example! We have a Carousel component design in development, and will be releasing it in the near future as a new Root widget! It will support both Card-like items and unbounded items, with built in nav/scrolling for desktop/mobile.

[tylersmith-openai]

The main challenge I’m facing with the current system is the limited charting capabilities. I work extensively with pricing data, and users often want to “chat with their data.” Right now, the Recharts library feels restrictive compared to something like ECharts, which offers much richer visualization options.

Got it - thank you! Makes total sense, we definitely need better support for more complex data visualization. I'll follow-up here when I have more to share about how to unblock you on this.

I can say that I don't believe our solution will involve rendering arbitrary UI as "1p" inside ChatKit, but that we are likely to provide escape hatches through an Iframe widget component (similar to ChatGPT Apps SDK-style isolated apps).

Our intended usage for this would be complex UI visualizations, though, and not for bringing your design system into ChatKit. Our solution for matching styles will continue to be our theme property, where our goal is to get incredibly close, but retain enough control that we are 100% confident in how to build UI based on those values, introduce new UI/UX, etc.

[MatthieuBonnardot]

Totally understandable, Exciting ! Thank you very much @tylersmith-openai !