-
Notifications
You must be signed in to change notification settings - Fork 129
Description
I have recently been doing a large update of packages in integrations and have found in that process that the errors that are printed could be improved.
Two example cases:
- Invalid field type
1. file "/Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/gcp-2.12.0.zip/data_stream/audit/fields/fields.yml" is invalid: field 0.fields.2.type: 0.fields.2.type must be one of the following: "alias", "histogram", "constant_keyword", "text", "match_only_text", "keyword", "long", "integer", "short", "byte", "double", "float", "half_float", "scaled_float", "date", "date_nanos", "boolean", "binary", "integer_range", "float_range", "long_range", "double_range", "date_range", "ip_range", "group", "geo_point", "object", "ip", "nested", "flattened", "wildcard", "version", "unsigned_long" - Duplicated field definition
1. field "cloud.instance.id" is defined multiple times for data stream "storage", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/gcp-2.12.0.zip/data_stream/storage/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/gcp-2.12.0.zip/data_stream/storage/fields/ecs.yml
In both cases there is unnecessary information that reduces the readability of the error, particularly when there are many lines of errors. So I would suggest that the file paths be made relative to the package at least, and probably to the data stream.
In the case of the invalid field, the value that is invalid is not displayed, only the values that it may take. This in conjunction with the fact that the name of the field is not printed, only its position in the AST, make it difficult to find the offending value. In the case above it is not particularly hard since it is early in the file, but some are quite deep. So I would suggest that the offending value be printed, the name of the field (i.e. the value of the name field in the yml) and the file position also be provided (e.g. audit/fields/fields.yml:21:13 in the case above) to allow direct movement to the error in an editor.
The impact of the current output format can be clearly demonstrated with the following detail.
Error: building package failed: invalid content found in built zip package: found 87 validation errors: 1. field "host.ip" is defined multiple times for data stream "rfb", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/rfb/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/rfb/fields/ecs.yml 2. field "container.id" is defined multiple times for data stream "rfb", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/rfb/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/rfb/fields/beats.yml 3. field "container.id" is defined multiple times for data stream "smb_cmd", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smb_cmd/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smb_cmd/fields/beats.yml 4. field "host.ip" is defined multiple times for data stream "smb_cmd", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smb_cmd/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smb_cmd/fields/ecs.yml 5. field "container.id" is defined multiple times for data stream "smb_files", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smb_files/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smb_files/fields/beats.yml 6. field "host.ip" is defined multiple times for data stream "smb_files", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smb_files/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smb_files/fields/ecs.yml 7. field "host.ip" is defined multiple times for data stream "smb_mapping", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smb_mapping/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smb_mapping/fields/ecs.yml 8. field "container.id" is defined multiple times for data stream "smb_mapping", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smb_mapping/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smb_mapping/fields/beats.yml 9. field "container.id" is defined multiple times for data stream "dnp3", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dnp3/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dnp3/fields/beats.yml 10. field "host.ip" is defined multiple times for data stream "dnp3", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dnp3/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dnp3/fields/ecs.yml 11. field "host.ip" is defined multiple times for data stream "ntlm", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ntlm/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ntlm/fields/ecs.yml 12. field "container.id" is defined multiple times for data stream "ntlm", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ntlm/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ntlm/fields/beats.yml 13. field "container.id" is defined multiple times for data stream "rdp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/rdp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/rdp/fields/beats.yml 14. field "host.ip" is defined multiple times for data stream "rdp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/rdp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/rdp/fields/ecs.yml 15. field "host.ip" is defined multiple times for data stream "http", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/http/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/http/fields/ecs.yml 16. field "container.id" is defined multiple times for data stream "http", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/http/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/http/fields/beats.yml 17. field "host.ip" is defined multiple times for data stream "known_certs", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/known_certs/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/known_certs/fields/ecs.yml 18. field "container.id" is defined multiple times for data stream "known_certs", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/known_certs/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/known_certs/fields/beats.yml 19. field "host.ip" is defined multiple times for data stream "modbus", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/modbus/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/modbus/fields/ecs.yml 20. field "container.id" is defined multiple times for data stream "modbus", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/modbus/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/modbus/fields/beats.yml 21. field "host.ip" is defined multiple times for data stream "mysql", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/mysql/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/mysql/fields/ecs.yml 22. field "container.id" is defined multiple times for data stream "mysql", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/mysql/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/mysql/fields/beats.yml 23. field "host.ip" is defined multiple times for data stream "ntp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ntp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ntp/fields/ecs.yml 24. field "container.id" is defined multiple times for data stream "ntp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ntp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ntp/fields/beats.yml 25. field "container.id" is defined multiple times for data stream "dhcp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dhcp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dhcp/fields/beats.yml 26. field "host.ip" is defined multiple times for data stream "dhcp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dhcp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dhcp/fields/ecs.yml 27. field "host.ip" is defined multiple times for data stream "dns", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dns/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dns/fields/ecs.yml 28. field "container.id" is defined multiple times for data stream "dns", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dns/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dns/fields/beats.yml 29. field "host.ip" is defined multiple times for data stream "files", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/files/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/files/fields/ecs.yml 30. field "container.id" is defined multiple times for data stream "files", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/files/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/files/fields/beats.yml 31. field "host.ip" is defined multiple times for data stream "socks", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/socks/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/socks/fields/ecs.yml 32. field "container.id" is defined multiple times for data stream "socks", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/socks/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/socks/fields/beats.yml 33. field "host.ip" is defined multiple times for data stream "tunnel", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/tunnel/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/tunnel/fields/ecs.yml 34. field "container.id" is defined multiple times for data stream "tunnel", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/tunnel/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/tunnel/fields/beats.yml 35. field "host.ip" is defined multiple times for data stream "ftp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ftp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ftp/fields/ecs.yml 36. field "container.id" is defined multiple times for data stream "ftp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ftp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ftp/fields/beats.yml 37. field "host.ip" is defined multiple times for data stream "radius", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/radius/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/radius/fields/ecs.yml 38. field "container.id" is defined multiple times for data stream "radius", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/radius/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/radius/fields/beats.yml 39. field "host.ip" is defined multiple times for data stream "smtp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smtp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smtp/fields/ecs.yml 40. field "container.id" is defined multiple times for data stream "smtp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smtp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/smtp/fields/beats.yml 41. field "host.ip" is defined multiple times for data stream "signature", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/signature/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/signature/fields/ecs.yml 42. field "container.id" is defined multiple times for data stream "signature", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/signature/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/signature/fields/beats.yml 43. field "container.id" is defined multiple times for data stream "ssl", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ssl/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ssl/fields/beats.yml 44. field "tls.version_protocol" is defined multiple times for data stream "ssl", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ssl/fields/ecs.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ssl/fields/ecs.yml 45. field "host.ip" is defined multiple times for data stream "ssl", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ssl/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ssl/fields/ecs.yml 46. field "container.id" is defined multiple times for data stream "syslog", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/syslog/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/syslog/fields/beats.yml 47. field "host.ip" is defined multiple times for data stream "syslog", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/syslog/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/syslog/fields/ecs.yml 48. field "host.ip" is defined multiple times for data stream "traceroute", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/traceroute/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/traceroute/fields/ecs.yml 49. field "container.id" is defined multiple times for data stream "traceroute", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/traceroute/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/traceroute/fields/beats.yml 50. field "host.ip" is defined multiple times for data stream "intel", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/intel/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/intel/fields/ecs.yml 51. field "container.id" is defined multiple times for data stream "intel", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/intel/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/intel/fields/beats.yml 52. field "container.id" is defined multiple times for data stream "known_hosts", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/known_hosts/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/known_hosts/fields/beats.yml 53. field "host.ip" is defined multiple times for data stream "known_hosts", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/known_hosts/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/known_hosts/fields/ecs.yml 54. field "host.ip" is defined multiple times for data stream "ocsp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ocsp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ocsp/fields/ecs.yml 55. field "container.id" is defined multiple times for data stream "ocsp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ocsp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ocsp/fields/beats.yml 56. field "container.id" is defined multiple times for data stream "irc", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/irc/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/irc/fields/beats.yml 57. field "host.ip" is defined multiple times for data stream "irc", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/irc/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/irc/fields/ecs.yml 58. field "container.id" is defined multiple times for data stream "kerberos", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/kerberos/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/kerberos/fields/beats.yml 59. field "host.ip" is defined multiple times for data stream "kerberos", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/kerberos/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/kerberos/fields/ecs.yml 60. field "container.id" is defined multiple times for data stream "sip", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/sip/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/sip/fields/beats.yml 61. field "host.ip" is defined multiple times for data stream "sip", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/sip/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/sip/fields/ecs.yml 62. field "container.id" is defined multiple times for data stream "connection", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/connection/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/connection/fields/beats.yml 63. field "host.ip" is defined multiple times for data stream "connection", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/connection/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/connection/fields/ecs.yml 64. field "container.id" is defined multiple times for data stream "dpd", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dpd/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dpd/fields/beats.yml 65. field "host.ip" is defined multiple times for data stream "dpd", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dpd/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dpd/fields/ecs.yml 66. field "container.id" is defined multiple times for data stream "known_services", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/known_services/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/known_services/fields/beats.yml 67. field "host.ip" is defined multiple times for data stream "known_services", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/known_services/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/known_services/fields/ecs.yml 68. field "host.ip" is defined multiple times for data stream "pe", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/pe/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/pe/fields/ecs.yml 69. field "container.id" is defined multiple times for data stream "pe", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/pe/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/pe/fields/beats.yml 70. field "container.id" is defined multiple times for data stream "ssh", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ssh/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ssh/fields/beats.yml 71. field "host.ip" is defined multiple times for data stream "ssh", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ssh/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/ssh/fields/ecs.yml 72. field "host.ip" is defined multiple times for data stream "snmp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/snmp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/snmp/fields/ecs.yml 73. field "container.id" is defined multiple times for data stream "snmp", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/snmp/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/snmp/fields/beats.yml 74. field "container.id" is defined multiple times for data stream "software", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/software/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/software/fields/beats.yml 75. field "host.ip" is defined multiple times for data stream "software", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/software/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/software/fields/ecs.yml 76. field "container.id" is defined multiple times for data stream "stats", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/stats/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/stats/fields/beats.yml 77. field "host.ip" is defined multiple times for data stream "stats", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/stats/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/stats/fields/ecs.yml 78. field "host.ip" is defined multiple times for data stream "weird", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/weird/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/weird/fields/ecs.yml 79. field "container.id" is defined multiple times for data stream "weird", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/weird/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/weird/fields/beats.yml 80. field "host.ip" is defined multiple times for data stream "x509", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/x509/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/x509/fields/ecs.yml 81. field "container.id" is defined multiple times for data stream "x509", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/x509/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/x509/fields/beats.yml 82. field "container.id" is defined multiple times for data stream "capture_loss", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/capture_loss/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/capture_loss/fields/beats.yml 83. field "host.ip" is defined multiple times for data stream "capture_loss", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/capture_loss/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/capture_loss/fields/ecs.yml 84. field "host.ip" is defined multiple times for data stream "dce_rpc", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dce_rpc/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dce_rpc/fields/ecs.yml 85. field "container.id" is defined multiple times for data stream "dce_rpc", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dce_rpc/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/dce_rpc/fields/beats.yml 86. field "container.id" is defined multiple times for data stream "notice", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/notice/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/notice/fields/beats.yml 87. field "host.ip" is defined multiple times for data stream "notice", found in: /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/notice/fields/agent.yml, /Users/xxxxxxxx/go/src/github.com/elastic/integrations/build/packages/zeek-2.6.0.zip/data_stream/notice/fields/ecs.yml
This would be made much clearer if the path were made relative to the data stream, file positions were emitted and the outputs were made to be consistent between runs (preferably grouped by file in sections in lexical order of datastream name then file name grouped and then by position in file). A sketch of how this might look is in the following detail.
Error: building package failed: invalid content found in built zip package: found 87 validation errors: in data stream "capture_loss" "container.id" is defined multiple times capture_loss/fields/agent.yml:123:34 capture_loss/fields/beats.yml:123:34 "host.ip" is defined multiple times capture_loss/fields/agent.yml:123:34 capture_loss/fields/ecs.yml:123:34 in data stream "connection" "container.id" is defined multiple times connection/fields/agent.yml:123:34 connection/fields/beats.yml:123:34 "host.ip" is defined multiple times connection/fields/agent.yml:123:34 connection/fields/ecs.yml:123:34 in data stream "dce_rpc" "host.ip" is defined multiple times dce_rpc/fields/agent.yml:123:34 dce_rpc/fields/ecs.yml:123:34 "container.id" is defined multiple times dce_rpc/fields/agent.yml:123:34 dce_rpc/fields/beats.yml:123:34 in data stream "dhcp" "container.id" is defined multiple times dhcp/fields/agent.yml:123:34 dhcp/fields/beats.yml:123:34 "host.ip" is defined multiple times dhcp/fields/agent.yml:123:34 dhcp/fields/ecs.yml:123:34 in data stream "dnp3" "container.id" is defined multiple times dnp3/fields/agent.yml:123:34 dnp3/fields/beats.yml:123:34 "host.ip" is defined multiple times dnp3/fields/agent.yml:123:34 dnp3/fields/ecs.yml:123:34 in data stream "dns" "host.ip" is defined multiple times dns/fields/agent.yml:123:34 dns/fields/ecs.yml:123:34 "container.id" is defined multiple times dns/fields/agent.yml:123:34 dns/fields/beats.yml:123:34 in data stream "dpd" "container.id" is defined multiple times dpd/fields/agent.yml:123:34 dpd/fields/beats.yml:123:34 "host.ip" is defined multiple times dpd/fields/agent.yml:123:34 dpd/fields/ecs.yml:123:34 in data stream "files" "host.ip" is defined multiple times files/fields/agent.yml:123:34 files/fields/ecs.yml:123:34 "container.id" is defined multiple times files/fields/agent.yml:123:34 files/fields/beats.yml:123:34 in data stream "ftp" "host.ip" is defined multiple times ftp/fields/agent.yml:123:34 ftp/fields/ecs.yml:123:34 "container.id" is defined multiple times ftp/fields/agent.yml:123:34 ftp/fields/beats.yml:123:34 in data stream "http" "host.ip" is defined multiple times http/fields/agent.yml:123:34 http/fields/ecs.yml:123:34 "container.id" is defined multiple times http/fields/agent.yml:123:34 http/fields/beats.yml:123:34 in data stream "intel" "host.ip" is defined multiple times intel/fields/agent.yml:123:34 intel/fields/ecs.yml:123:34 "container.id" is defined multiple times intel/fields/agent.yml:123:34 intel/fields/beats.yml:123:34 in data stream "irc" "container.id" is defined multiple times irc/fields/agent.yml:123:34 irc/fields/beats.yml:123:34 "host.ip" is defined multiple times irc/fields/agent.yml:123:34 irc/fields/ecs.yml:123:34 in data stream "kerberos" "container.id" is defined multiple times kerberos/fields/agent.yml:123:34 kerberos/fields/beats.yml:123:34 "host.ip" is defined multiple times kerberos/fields/agent.yml:123:34 kerberos/fields/ecs.yml:123:34 in data stream "known_certs" "host.ip" is defined multiple times known_certs/fields/agent.yml:123:34 known_certs/fields/ecs.yml:123:34 "container.id" is defined multiple times known_certs/fields/agent.yml:123:34 known_certs/fields/beats.yml:123:34 in data stream "known_hosts" "container.id" is defined multiple times known_hosts/fields/agent.yml:123:34 known_hosts/fields/beats.yml:123:34 "host.ip" is defined multiple times known_hosts/fields/agent.yml:123:34 known_hosts/fields/ecs.yml:123:34 in data stream "known_services" "container.id" is defined multiple times known_services/fields/agent.yml:123:34 known_services/fields/beats.yml:123:34 "host.ip" is defined multiple times known_services/fields/agent.yml:123:34 known_services/fields/ecs.yml:123:34 in data stream "modbus" "host.ip" is defined multiple times modbus/fields/agent.yml:123:34 modbus/fields/ecs.yml:123:34 "container.id" is defined multiple times modbus/fields/agent.yml:123:34 modbus/fields/beats.yml:123:34 in data stream "mysql" "host.ip" is defined multiple times mysql/fields/agent.yml:123:34 mysql/fields/ecs.yml:123:34 "container.id" is defined multiple times mysql/fields/agent.yml:123:34 mysql/fields/beats.yml:123:34 in data stream "notice" "container.id" is defined multiple times notice/fields/agent.yml:123:34 notice/fields/beats.yml:123:34 "host.ip" is defined multiple times notice/fields/agent.yml:123:34 notice/fields/ecs.yml:123:34 in data stream "ntlm" "host.ip" is defined multiple times ntlm/fields/agent.yml:123:34 ntlm/fields/ecs.yml:123:34 "container.id" is defined multiple times ntlm/fields/agent.yml:123:34 ntlm/fields/beats.yml:123:34 in data stream "ntp" "host.ip" is defined multiple times ntp/fields/agent.yml:123:34 ntp/fields/ecs.yml:123:34 "container.id" is defined multiple times ntp/fields/agent.yml:123:34 ntp/fields/beats.yml:123:34 in data stream "ocsp" "host.ip" is defined multiple times ocsp/fields/agent.yml:123:34 ocsp/fields/ecs.yml:123:34 "container.id" is defined multiple times ocsp/fields/agent.yml:123:34 ocsp/fields/beats.yml:123:34 in data stream "pe" "host.ip" is defined multiple times pe/fields/agent.yml:123:34 pe/fields/ecs.yml:123:34 "container.id" is defined multiple times pe/fields/agent.yml:123:34 pe/fields/beats.yml:123:34 in data stream "radius" "host.ip" is defined multiple times radius/fields/agent.yml:123:34 radius/fields/ecs.yml:123:34 "container.id" is defined multiple times radius/fields/agent.yml:123:34 radius/fields/beats.yml:123:34 in data stream "rdp" "container.id" is defined multiple times rdp/fields/agent.yml:123:34 rdp/fields/beats.yml:123:34 "host.ip" is defined multiple times rdp/fields/agent.yml:123:34 rdp/fields/ecs.yml:123:34 in data stream "rfb" "host.ip" is defined multiple times rfb/fields/agent.yml:123:34 rfb/fields/ecs.yml:123:34 "container.id" is defined multiple times rfb/fields/agent.yml:123:34 rfb/fields/beats.yml:123:34 in data stream "signature" "host.ip" is defined multiple times signature/fields/agent.yml:123:34 signature/fields/ecs.yml:123:34 "container.id" is defined multiple times signature/fields/agent.yml:123:34 signature/fields/beats.yml:123:34 in data stream "sip" "container.id" is defined multiple times sip/fields/agent.yml:123:34 sip/fields/beats.yml:123:34 "host.ip" is defined multiple times sip/fields/agent.yml:123:34 sip/fields/ecs.yml:123:34 in data stream "smb_cmd" "container.id" is defined multiple times smb_cmd/fields/agent.yml:123:34 smb_cmd/fields/beats.yml:123:34 "host.ip" is defined multiple times smb_cmd/fields/agent.yml:123:34 smb_cmd/fields/ecs.yml:123:34 in data stream "smb_files" "container.id" is defined multiple times smb_files/fields/agent.yml:123:34 smb_files/fields/beats.yml:123:34 "host.ip" is defined multiple times smb_files/fields/agent.yml:123:34 smb_files/fields/ecs.yml:123:34 in data stream "smb_mapping" "host.ip" is defined multiple times smb_mapping/fields/agent.yml:123:34 smb_mapping/fields/ecs.yml:123:34 "container.id" is defined multiple times smb_mapping/fields/agent.yml:123:34 smb_mapping/fields/beats.yml:123:34 in data stream "smtp" "host.ip" is defined multiple times smtp/fields/agent.yml:123:34 smtp/fields/ecs.yml:123:34 "container.id" is defined multiple times smtp/fields/agent.yml:123:34 smtp/fields/beats.yml:123:34 in data stream "snmp" "host.ip" is defined multiple times snmp/fields/agent.yml:123:34 snmp/fields/ecs.yml:123:34 "container.id" is defined multiple times snmp/fields/agent.yml:123:34 snmp/fields/beats.yml:123:34 in data stream "socks" "host.ip" is defined multiple times socks/fields/agent.yml:123:34 socks/fields/ecs.yml:123:34 "container.id" is defined multiple times socks/fields/agent.yml:123:34 socks/fields/beats.yml:123:34 in data stream "software" "container.id" is defined multiple times software/fields/agent.yml:123:34 software/fields/beats.yml:123:34 "host.ip" is defined multiple times software/fields/agent.yml:123:34 software/fields/ecs.yml:123:34 in data stream "ssh" "container.id" is defined multiple times ssh/fields/agent.yml:123:34 ssh/fields/beats.yml:123:34 "host.ip" is defined multiple times ssh/fields/agent.yml:123:34 ssh/fields/ecs.yml:123:34 in data stream "ssl" "container.id" is defined multiple times ssl/fields/agent.yml:123:34 ssl/fields/beats.yml:123:34 "tls.version_protocol" is defined multiple times ssl/fields/ecs.yml:123:34 ssl/fields/ecs.yml:123:34 "host.ip" is defined multiple times ssl/fields/agent.yml:123:34 ssl/fields/ecs.yml:123:34 in data stream "stats" "container.id" is defined multiple times stats/fields/agent.yml:123:34 stats/fields/beats.yml:123:34 "host.ip" is defined multiple times stats/fields/agent.yml:123:34 stats/fields/ecs.yml:123:34 in data stream "syslog" "container.id" is defined multiple times syslog/fields/agent.yml:123:34 syslog/fields/beats.yml:123:34 "host.ip" is defined multiple times syslog/fields/agent.yml:123:34 syslog/fields/ecs.yml:123:34 in data stream "traceroute" "host.ip" is defined multiple times traceroute/fields/agent.yml:123:34 traceroute/fields/ecs.yml:123:34 "container.id" is defined multiple times traceroute/fields/agent.yml:123:34 traceroute/fields/beats.yml:123:34 in data stream "tunnel" "host.ip" is defined multiple times tunnel/fields/agent.yml:123:34 tunnel/fields/ecs.yml:123:34 "container.id" is defined multiple times tunnel/fields/agent.yml:123:34 tunnel/fields/beats.yml:123:34 in data stream "weird" "host.ip" is defined multiple times weird/fields/agent.yml:123:34 weird/fields/ecs.yml:123:34 "container.id" is defined multiple times weird/fields/agent.yml:123:34 weird/fields/beats.yml:123:34 in data stream "x509" "host.ip" is defined multiple times x509/fields/agent.yml:123:34 x509/fields/ecs.yml:123:34 "container.id" is defined multiple times x509/fields/agent.yml:123:34 x509/fields/beats.yml:123:34
It is easier to read, follows the ordering of the files in an editor's file pane and the fields are presented in the order that you would find them in the files as they are being edited (not done in the example here but is straighforward to do mechanically).