[Azure App Service] New dashboard

[giorgi-imerlishvili-elastic]

Proposed commit message

Enhancement: Add dashboard for log categories Application Logs, Audit Logs, Console Logs, HTTP Logs, IPsec Audit Logs and Platform Logs.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Screenshots

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[Linu-Elias]

You can add the screenshots attribute in the manifest file and include references to the dashboard image, similar to how it's done in other integrations.

[mykola-elastic]

Yes, please make sure screenshots contain the whole dashboard, and also please add those same screenshots to the PR page (right now the screenshot in the PR description is truncated)

[giorgi-imerlishvili-elastic]

done

[Linu-Elias]

Instead of having multiple changelog entries for the same PR, you can consolidate them into a single entry with a combined description.
example for reference:

integrations/packages/ti_abusech/changelog.yml

Line 4 in d8a5e6d

- description: |

[giorgi-imerlishvili-elastic]

I wanted to have in one change log as well. I like combined description idea, but what about type it can only have enhancement or bugfix value as I know. This PR contains enhancement and also bugfixes...

[mykola-elastic]

will it be hard to split the changes into 2 separate PRs? or even 3 as I see 3 changelogs?

Is it OK to have several changelogs in one PR @ishleenk17 ?

[muthu-mps]

@giorgi-imerlishvili-elastic - Could we create a separate PR for the ingest pipeline changes? Since we're adding a new dashboard to the integrations, it would be better to keep these changes distinct.

[giorgi-imerlishvili-elastic]

Removed bugfixes from this PR and created new one: #15591

[giorgi-imerlishvili-elastic]

Removed log level panels from IPSecAudit and Audit logs section and updated screenshots

Any ideas, what else information/panel can be added instead of those? @lucian-ioan @muthu-mps

[muthu-mps]

Can we update the dashboard image with real time log data. The current one shows 0 in all the panels.

[lucian-ioan]

Yeah I confirm, just double checked and IPSecAuditLogs and AuditLogs doesn't have log level field 😕, I'll remove panels for those.

I believe it's also the case for AppServiceHTTPLogs.

Any ideas, what else information/panel can be added instead of those? @lucian-ioan @muthu-mps

The goal is to try to provide value for the users. For example for the ones without Log Level:

1. IpSecAuditLogs -> Fields: client_ip and result. Result can only be "Allowed" or "Denied". Top IPs denied would help understand which IPs keep trying to gain access without authorization.

2. AppServiceAuditLogs -> Fields: user and protocol. Top Active Users and Top Protocol which tells the most popular auth methods (ex. SSH, OAuth2)

3. AppServiceHTTPLogs -> Fields: sc_status = HTTP response code, time_taken = latency, cs_uri_stem = endpoint. Very useful fields here, starting from Average Latency, Status Codes Pie Chart, Top Endpoints Used all the way to aggregations such as highest average latency by endpoint.

[Linu-Elias]

1. Please dismiss the message at the top and then capture the screenshot.
2. It should be "Total Errors" instead of "Total errors" so that the titles are consistent.
3. Refer to this Observability integrations dashboards best practices to see if we missed anything.

[giorgi-imerlishvili-elastic]

All feedbacks were addressed please review and let me know if I missed something @lucian-ioan, @muthu-mps, @Linu-Elias

[muthu-mps]

@daniela-elastic - Can you take a look into the dashboard changes?

[lucian-ioan]

1. I'd rename any "Top 10 values of" with just the actual field title and any other raw fields with simpler names.
2. Average Response Time should have a suffix (ms) to be more readable.

[gpop63]

@giorgi-imerlishvili-elastic here are some dashboard guidelines you can follow https://docs.google.com/document/d/1aTqkl0BSYPa1WeKXkKCChQaIqM0J5w4vY-8flHDUj8A/edit?tab=t.0#heading=h.qt35o6gmkwy8

[daniela-elastic]

Please address the following first. No need for second review after you fix it so conditionally approving to save time:

1. what does "Activity" refer to? Is that number of logs or something else? If it's number of logs can you say something like "Activity - number of logs"? what do i see if i click on the "i" icon next to "Activity"? Perhaps this could be used to provide the additional info needed
2. X-axis name - "per 12 hours" - can you check with @ishleenk17 if this is how we want to describe the x-axis? ALso, even though you say it's per 12 hours, i can see that you're actually showing 24 hours so it's a bit confusing

[ishleenk17]

Reviewing from the screenshot in the PR:

1. The total errors is such elongated box and others are all v compact. Can we have a better alignment there.
   Something like this screenshot. 1 under the other ?

2. Can we have the (i) icon next to log level panels ?

2. X-axis name - "per 12 hours" - can you check with @ishleenk17 if this is how we want to describe the x-axis? ALso, even though you say it's per 12 hours, i can see that you're actually showing 24 hours so it's a bit confusing

@giorgi-imerlishvili-elastic : Is your x axis the @timestamp field ? the per 12 hours, per 24 hours changes as per what time range we choose above. We usually do something like below.

[giorgi-imerlishvili-elastic]

The total errors is such elongated box and others are all v compact. Can we have a better alignment there.
Something like this screenshot. 1 under the other ?

I'll try make some changes

@giorgi-imerlishvili-elastic : Is your x axis the @timestamp field ?

Yes it's @timestamp field

[muthu-mps]

1. I'd rename any "Top 10 values of" with just the actual field title and any other raw fields with simpler names.
2. Average Response Time should have a suffix (ms) to be more readable.

@lucian-ioan - If this comment is addressed, Can you approve it?

[ishleenk17]

Log level panel doesn't have mention of @timestamp field like other panels/ OTherwsise looks good

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 8e86e91

History

• 💚 Build #33346 succeeded 8042fc4
• 💚 Build #33342 succeeded 22f75d2
• 💚 Build #33318 succeeded 342631b
• 💚 Build #33100 succeeded 86dab2b
• 💚 Build #32927 succeeded b9dc050
• 💚 Build #32649 succeeded c675251

cc @giorgi-imerlishvili-elastic

[elastic-vault-github-plugin-prod]

Package azure_app_service - 0.8.0 containing this change is available at https://epr.elastic.co/package/azure_app_service/0.8.0/