Migrate API Key Mgmt UI tests to Scout #241510
Conversation
|
🤖 Jobs for this PR can be triggered through checkboxes. 🚧
ℹ️ To trigger the CI, please tick the checkbox below 👇
|
There was a problem hiding this comment.
Pull Request Overview
This PR migrates API Keys functional tests from the legacy FTR (Functional Test Runner) framework to Scout (Playwright-based testing framework). The migration removes FTR-specific test files and page objects, replacing them with Scout-compatible implementations that leverage Playwright APIs and modern testing patterns.
Key changes:
- Removed FTR-based tests and page objects for API Keys
- Created new Scout/Playwright-based tests with equivalent coverage
- Added Scout API service for API Keys management with comprehensive CRUD operations
- Enhanced accessibility with
announceOnMountprops on callout components in the UI
Reviewed Changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 18 comments.
Show a summary per file
| File | Description |
|---|---|
x-pack/platform/test/functional/page_objects/index.ts |
Removed FTR page object provider for API Keys |
x-pack/platform/test/functional/page_objects/api_keys_page.ts |
Deleted FTR-based page object implementation |
x-pack/platform/test/functional/apps/api_keys/scout/ui/tests/home_page.spec.ts |
New Scout test covering API key creation, updating, deletion, and querying |
x-pack/platform/test/functional/apps/api_keys/scout/ui/tests/api_keys_security.spec.ts |
New Scout test for security permissions and access control |
x-pack/platform/test/functional/apps/api_keys/scout/ui/playwright.config.ts |
Scout/Playwright configuration file |
x-pack/platform/test/functional/apps/api_keys/scout/ui/fixtures/page_objects/index.ts |
Scout page object extension mechanism |
x-pack/platform/test/functional/apps/api_keys/scout/ui/fixtures/page_objects/api_keys_page.ts |
Scout-compatible page object with Playwright locators |
x-pack/platform/test/functional/apps/api_keys/scout/ui/fixtures/index.ts |
Scout test fixture definitions |
x-pack/platform/test/functional/apps/api_keys/index.ts |
Deleted FTR test suite index |
x-pack/platform/test/functional/apps/api_keys/home_page.ts |
Deleted FTR test implementation |
x-pack/platform/test/functional/apps/api_keys/feature_controls/api_keys_security.ts |
Deleted FTR security test |
x-pack/platform/test/functional/apps/api_keys/config.ts |
Deleted FTR config file |
x-pack/platform/test/functional/apps/api_keys/api_keys_helpers.ts |
Deleted FTR helper utilities |
x-pack/platform/packages/shared/security/api_key_management/src/components/api_key_flyout.tsx |
Added test subject and accessibility announcements to flyout |
src/platform/packages/shared/kbn-scout/src/playwright/fixtures/scope/worker/apis/index.ts |
Registered API Keys service in Scout API services |
src/platform/packages/shared/kbn-scout/src/playwright/fixtures/scope/worker/apis/api_keys/types.ts |
TypeScript type definitions for API Keys API |
src/platform/packages/shared/kbn-scout/src/playwright/fixtures/scope/worker/apis/api_keys/index.ts |
API Keys service implementation with CRUD and cleanup utilities |
src/platform/packages/shared/kbn-scout/src/playwright/fixtures/scope/worker/apis/api_keys/README.md |
Comprehensive documentation for API Keys service usage |
| */ | ||
| async waitForSubmitButtonOnApiKeyFlyoutEnabled() { | ||
| await this.formFlyoutSubmitButton.waitFor({ state: 'visible' }); | ||
| await this.page.waitForTimeout(10000); // Wait up to 10 seconds for the button to be enabled |
There was a problem hiding this comment.
Using a fixed waitForTimeout(10000) is an anti-pattern that makes tests slower and more flaky. Instead, use Playwright's built-in waiting mechanism waitFor({ state: 'enabled' }) or a poll-based approach to wait for the submit button to be enabled. This will make the test complete as soon as the button is enabled rather than always waiting 10 seconds.
| await this.page.waitForTimeout(10000); // Wait up to 10 seconds for the button to be enabled | |
| await this.formFlyoutSubmitButton.waitFor({ state: 'enabled' }); |
| await pageObjects.apiKeys.clickSubmitButtonOnApiKeyFlyout(); | ||
|
|
||
| // Wait for modal to close | ||
| await page.waitForTimeout(1000); |
There was a problem hiding this comment.
Using arbitrary waitForTimeout(1000) is a test anti-pattern. Instead of waiting a fixed 1 second, wait for a specific condition (e.g., modal closing, URL change, or element detachment) using Playwright's waitFor methods. This makes tests faster and more reliable.
| await page.waitForTimeout(1000); | |
| // Replace arbitrary timeout with waiting for modal to be detached | |
| await page.waitForSelector(pageObjects.apiKeys.apiKeyModalSelector, { state: 'detached' }); |
| await pageObjects.apiKeys.clickSubmitButtonOnApiKeyFlyout(); | ||
|
|
||
| // Wait for modal to close | ||
| await page.waitForTimeout(1000); |
There was a problem hiding this comment.
Using arbitrary waitForTimeout(1000) is a test anti-pattern. Instead of waiting a fixed 1 second, wait for a specific condition using Playwright's waitFor methods.
| await page.waitForTimeout(1000); | |
| await expect.poll(async () => await pageObjects.apiKeys.isApiKeyModalExists()).toBe(false); |
|
|
||
| // Refresh browser | ||
| await page.reload(); | ||
| await page.waitForTimeout(1000); |
There was a problem hiding this comment.
Using arbitrary waitForTimeout(1000) is a test anti-pattern. Replace with a proper wait condition.
| await page.waitForTimeout(1000); | |
| // Wait for the API Keys page to be loaded after reload | |
| await pageObjects.apiKeys.waitForPageLoaded(); |
| test('active/expired filter buttons work as expected', async ({ page, pageObjects }) => { | ||
| // Click active filter | ||
| await pageObjects.apiKeys.clickExpiryFilters('active'); | ||
| await page.waitForTimeout(500); |
There was a problem hiding this comment.
Multiple instances of arbitrary waitForTimeout() calls throughout the test. These should be replaced with proper Playwright wait conditions that check for specific states (e.g., network idle, element visibility) rather than arbitrary delays.
|
|
||
| // Deselect custom role | ||
| await page.testSubj.click(`userProfileSelectableOption-elastic`); | ||
| await page.waitForTimeout(500); |
There was a problem hiding this comment.
Extensive use of arbitrary waitForTimeout() delays throughout the querying tests. Consider refactoring to use proper wait conditions or adding a helper method that waits for filter application to complete (e.g., waiting for network requests to finish or checking for specific UI state changes).
|
|
||
| // Click elastic option | ||
| await page.testSubj.click(`userProfileSelectableOption-elastic_${samlAuth.customRoleName}`); | ||
| await page.waitForTimeout(500); |
There was a problem hiding this comment.
Extensive use of arbitrary waitForTimeout() delays throughout the querying tests. Consider refactoring to use proper wait conditions or adding a helper method that waits for filter application to complete (e.g., waiting for network requests to finish or checking for specific UI state changes).
| test('search bar works as expected', async ({ page, pageObjects }) => { | ||
| // Search for custom role api key | ||
| await pageObjects.apiKeys.setSearchBarValue('custom role api key'); | ||
| await page.waitForTimeout(500); |
There was a problem hiding this comment.
Extensive use of arbitrary waitForTimeout() delays throughout the querying tests. Consider refactoring to use proper wait conditions or adding a helper method that waits for filter application to complete (e.g., waiting for network requests to finish or checking for specific UI state changes).
| await page.waitForTimeout(500); | ||
|
|
||
| // Ensure API key exists | ||
| await pageObjects.apiKeys.ensureApiKeyExists('custom role api key'); | ||
|
|
||
| // Search with exact match quotes | ||
| await pageObjects.apiKeys.setSearchBarValue('"custom role api key"'); | ||
| await page.waitForTimeout(500); | ||
|
|
||
| // Ensure API key exists | ||
| await pageObjects.apiKeys.ensureApiKeyExists('my api key'); | ||
|
|
||
| // Search with partial match | ||
| await pageObjects.apiKeys.setSearchBarValue('"api"'); | ||
| await page.waitForTimeout(500); |
There was a problem hiding this comment.
Extensive use of arbitrary waitForTimeout() delays throughout the querying tests. Consider refactoring to use proper wait conditions or adding a helper method that waits for filter application to complete (e.g., waiting for network requests to finish or checking for specific UI state changes).
| await page.waitForTimeout(500); | |
| // Ensure API key exists | |
| await pageObjects.apiKeys.ensureApiKeyExists('custom role api key'); | |
| // Search with exact match quotes | |
| await pageObjects.apiKeys.setSearchBarValue('"custom role api key"'); | |
| await page.waitForTimeout(500); | |
| // Ensure API key exists | |
| await pageObjects.apiKeys.ensureApiKeyExists('my api key'); | |
| // Search with partial match | |
| await pageObjects.apiKeys.setSearchBarValue('"api"'); | |
| await page.waitForTimeout(500); | |
| // Ensure API key exists | |
| await pageObjects.apiKeys.ensureApiKeyExists('custom role api key'); | |
| // Search with exact match quotes | |
| await pageObjects.apiKeys.setSearchBarValue('"custom role api key"'); | |
| // Ensure API key exists | |
| await pageObjects.apiKeys.ensureApiKeyExists('my api key'); | |
| // Search with partial match | |
| await pageObjects.apiKeys.setSearchBarValue('"api"'); |
| await page.waitForTimeout(500); | ||
|
|
||
| // Ensure API key exists | ||
| await pageObjects.apiKeys.ensureApiKeyExists('custom role api key'); | ||
|
|
||
| // Search with exact match quotes | ||
| await pageObjects.apiKeys.setSearchBarValue('"custom role api key"'); | ||
| await page.waitForTimeout(500); | ||
|
|
||
| // Ensure API key exists | ||
| await pageObjects.apiKeys.ensureApiKeyExists('my api key'); | ||
|
|
||
| // Search with partial match | ||
| await pageObjects.apiKeys.setSearchBarValue('"api"'); | ||
| await page.waitForTimeout(500); |
There was a problem hiding this comment.
Extensive use of arbitrary waitForTimeout() delays throughout the querying tests. Consider refactoring to use proper wait conditions or adding a helper method that waits for filter application to complete (e.g., waiting for network requests to finish or checking for specific UI state changes).
| await page.waitForTimeout(500); | |
| // Ensure API key exists | |
| await pageObjects.apiKeys.ensureApiKeyExists('custom role api key'); | |
| // Search with exact match quotes | |
| await pageObjects.apiKeys.setSearchBarValue('"custom role api key"'); | |
| await page.waitForTimeout(500); | |
| // Ensure API key exists | |
| await pageObjects.apiKeys.ensureApiKeyExists('my api key'); | |
| // Search with partial match | |
| await pageObjects.apiKeys.setSearchBarValue('"api"'); | |
| await page.waitForTimeout(500); | |
| // Ensure API key exists | |
| await pageObjects.apiKeys.ensureApiKeyExists('custom role api key'); | |
| // Search with exact match quotes | |
| await pageObjects.apiKeys.setSearchBarValue('"custom role api key"'); | |
| // Ensure API key exists | |
| await pageObjects.apiKeys.ensureApiKeyExists('my api key'); | |
| // Search with partial match | |
| await pageObjects.apiKeys.setSearchBarValue('"api"'); |
💔 Build Failed
Failed CI StepsHistory
|
3 participants
Summary
Migrates API Key Management FTR tests to Scout, with the assistance of an AI Agent following instructions in https://github.com/elastic/kibana/tree/main/src/platform/packages/private/kbn-scout-info/llms.