[Observability:Streams] Manual ingest pipeline script validation

[couvq]

Closes https://github.com/elastic/observability-error-backlog/issues/314
Closes https://github.com/elastic/observability-error-backlog/issues/315
Closes https://github.com/elastic/observability-error-backlog/issues/317
Closes https://github.com/elastic/observability-error-backlog/issues/320

Description

• There are several Kibana error logs that are caused by script validation errors when making a direct call to the backend, we have been relying on elasticsearch to validate these up until this point, which leads to an error on the Kibana side. This PR adds a validation layer to our API and uses the simulation api to catch these validation errors before hitting elasticsearch. It also makes a change to our simulation handler to return a full error message so the UI includes a reason for failure to be more descriptive.

Screen recordings showing new behavior

UI

Screen.Recording.2025-12-08.at.4.25.57.PM.mov

API

Screen.Recording.2025-12-08.at.4.28.02.PM.mov

[elasticmachine]

🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

• Click to trigger kibana-pull-request for this PR!
• Click to trigger kibana-deploy-project-from-pr for this PR!
• Click to trigger kibana-deploy-cloud-from-pr for this PR!
• Click to trigger kibana-entity-store-performance-from-pr for this PR!

[couvq]

/ci

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 1bba4d5

Failed CI Steps

• Check Types