Release (#345)

* fix: poller username secrets (#336)

* fix: usernameSecrets nil pointer issue

* fix: update usernameSecrets structure

Co-authored-by: weliasz <[email protected]>
Co-authored-by: weliasz <[email protected]>

* fix: add option to change indexes names (#337)

* fix: add way to configure events and metrics index name and the documentation

* fix: add index names to splunk if clause

* chore(release): 1.3.1-beta.1

* add option to change indexes names ([#337](#337)) ([6cef754](6cef754))
* poller username secrets ([#336](#336)) ([7a3cae2](7a3cae2))

* fix: rework of enricher mongo data model (#334)

* fix: handle big enrichment objects

* feat: migration of shema

* feat: migration of schema

* feat: migration of schema

* feat: migration of schema

* tests: unit tests + refactoring

* tests: unit tests

* tests: fix to unit tests

* tests: fix to unit tests

* tests: fix to unit tests

* refactor: PR comments

* feat: more hosts per address (#333)

* chore: completed tests for inventory module

* feat: add a way to have more than one host per ip address

* test: add unit tests

* chore(release): 1.4.0-beta.1

# [1.4.0-beta.1](v1.3.1-beta.1...v1.4.0-beta.1) (2022-01-13)

### Bug Fixes

* rework of enricher mongo data model ([#334](#334)) ([bca5fb9](bca5fb9))

### Features

* more hosts per address ([#333](#333)) ([840815f](840815f))

* tests: more tests for custom task manager (#338)

* fix: correct if syntax

* chore: update lock for poetry

* fix: handle short varbind tables

Address issue with "No more variables left in this MIB View" appearing as attributes

* Update manager.py

* chore: bump minimum mib server

* feat: Load all standard mibs

When doing first walks or interacting with devices from new workers a substantial amount of time can be spent identifying and loading the standard mibs. This chance has each worker load standard mibs by default.

* chore: keep list of standard mibs

* chore: reset file

* Update pyproject.toml

* chore: reset versions

* chore(release): 1.4.0-beta.1

# [1.4.0-beta.1](v1.3.3...v1.4.0-beta.1) (2022-01-16)

### Bug Fixes

* add option to change indexes names ([#337](#337)) ([d2413bf](d2413bf))
* correct if syntax ([a0a4576](a0a4576))
* handle short varbind tables ([813f662](813f662))
* poller username secrets ([#336](#336)) ([6207c6f](6207c6f))
* rework of enricher mongo data model ([#334](#334)) ([41129cf](41129cf))

### Features

* Load all standard mibs ([5b6600c](5b6600c))
* more hosts per address ([#333](#333)) ([3d45c6a](3d45c6a))

* fix: bump

* feat: bump docs

* Revert "chore(release): 1.4.0-beta.1"

This reverts commit 447b427.

* chore(release): 1.4.0-beta.1

# [1.4.0-beta.1](v1.3.3...v1.4.0-beta.1) (2022-01-16)

### Bug Fixes

* add option to change indexes names ([#337](#337)) ([d2413bf](d2413bf))
* bump ([5ddab14](5ddab14))
* correct if syntax ([a0a4576](a0a4576))
* handle short varbind tables ([813f662](813f662))
* poller username secrets ([#336](#336)) ([6207c6f](6207c6f))
* rework of enricher mongo data model ([#334](#334)) ([41129cf](41129cf))

### Features

* bump docs ([021ad01](021ad01))
* Load all standard mibs ([5b6600c](5b6600c))
* more hosts per address ([#333](#333)) ([3d45c6a](3d45c6a))

* fix: bump

* Revert "chore(release): 1.4.0-beta.1"

This reverts commit f825bfb.

* chore(release): 1.4.0-beta.1

# [1.4.0-beta.1](v1.3.3...v1.4.0-beta.1) (2022-01-16)

### Bug Fixes

* add option to change indexes names ([#337](#337)) ([d2413bf](d2413bf))
* bump ([c6a94c5](c6a94c5))
* bump ([5ddab14](5ddab14))
* correct if syntax ([a0a4576](a0a4576))
* handle short varbind tables ([813f662](813f662))
* poller username secrets ([#336](#336)) ([6207c6f](6207c6f))
* rework of enricher mongo data model ([#334](#334)) ([41129cf](41129cf))

### Features

* bump docs ([021ad01](021ad01))
* Load all standard mibs ([5b6600c](5b6600c))
* more hosts per address ([#333](#333)) ([3d45c6a](3d45c6a))

### Reverts

* Revert "chore(release): 1.4.0-beta.1" ([8821a96](8821a96))

Co-authored-by: omrozowicz-splunk <[email protected]>
Co-authored-by: weliasz <[email protected]>
Co-authored-by: weliasz <[email protected]>
Co-authored-by: semantic-release-bot <[email protected]>

Author: 4 people

README.md

@@ -10,3 +10,6 @@ of components.
 # Documentation
 
 For deployment and user documentation [see](https://splunk-connect-for-snmp.readthedocs.io/)
+
+#
+#

charts/splunk-connect-for-snmp/Chart.lock

@@ -7,6 +7,6 @@ dependencies:
   version: 8.24.13
 - name: mibserver
   repository: https://pysnmp.github.io/mibs/charts/
-  version: 1.8.4
-digest: sha256:5eb0a97b13cf886ea4f19f3c6137c8907267266e0732cb0869d850bfaf84e763
-generated: "2022-01-15T21:12:33.542079-05:00"
+  version: 1.12.0
+digest: sha256:57c98330200019983930a73d12d21f627bbe8fde43d4a5ecb51b48165403326c
+generated: "2022-01-16T11:13:47.618842-05:00"

charts/splunk-connect-for-snmp/Chart.yaml

@@ -14,12 +14,12 @@ type: application
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
 
-version: 1.3.3
+version: 1.4.0-beta.1
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
 # It is recommended to use it with quotes.
-appVersion: "1.3.3"
+appVersion: "1.4.0-beta.1"
 #
 dependencies:
   - name: mongodb
@@ -29,7 +29,7 @@ dependencies:
     version: ~8.24
     repository: https://charts.bitnami.com/bitnami
   - name: mibserver
-    version: ~1.8.1
+    version: ~1.12.0
     repository: https://pysnmp.github.io/mibs/charts/
 #    condition: (optional) A yaml path that resolves to a boolean, used for enabling/disabling charts (e.g. subchart1.enabled )
 #    tags: # (optional)

charts/splunk-connect-for-snmp/templates/common/scheduler-inventory.yaml

@@ -5,7 +5,7 @@ metadata:
   labels:
     {{- include "splunk-connect-for-snmp.scheduler.labels" . | nindent 4 }}
 data:
-{{- with .Values.poller.inventory}}
+{{- with (.Values.poller).inventory}}
   inventory.csv: |-
 {{ . | indent 4}}
 {{- end }}

charts/splunk-connect-for-snmp/templates/inventory/job.yaml

@@ -39,6 +39,8 @@ spec:
             value: "http://{{ printf "%s-%s" .Release.Name "mibserver" }}/asn1/@mib@"
           - name: MIB_INDEX
             value: "http://{{ printf "%s-%s" .Release.Name "mibserver" }}/index.csv"
+          - name: MIB_STANDARD
+            value: "http://{{ printf "%s-%s" .Release.Name "mibserver" }}/standard.txt"
           - name: LOG_LEVEL
             value: {{ .Values.scheduler.logLevel | default "INFO" }}
           volumeMounts:
@@ -54,7 +56,7 @@ spec:
               - name: tmp
                 mountPath: "/tmp/"
                 readOnly: false                    
-              {{- if .Values.poller.usernameSecrets }}                 
+              {{- if (.Values.poller).usernameSecrets }}
               - name: snmpv3-secrets
                 mountPath: "/app/secrets/snmpv3"
                 readOnly: true
@@ -70,7 +72,7 @@ spec:
             items:
               - key: "config.yaml"
                 path: "config.yaml"
-        {{- if .Values.poller.usernameSecrets }}
+        {{- if (.Values.poller).usernameSecrets }}
         - name: snmpv3-secrets
           projected:
             sources:

charts/splunk-connect-for-snmp/templates/scheduler/deployment.yaml

@@ -48,6 +48,8 @@ spec:
               value: "http://{{ printf "%s-%s" .Release.Name "mibserver" }}/asn1/@mib@"
             - name: MIB_INDEX
               value: "http://{{ printf "%s-%s" .Release.Name "mibserver" }}/index.csv"
+            - name: MIB_STANDARD
+              value: "http://{{ printf "%s-%s" .Release.Name "mibserver" }}/standard.txt"
             - name: LOG_LEVEL
               value: {{ .Values.scheduler.logLevel | default "INFO" }}
             - name: INVENTORY_REFRESH_RATE
@@ -62,7 +64,7 @@ spec:
             - name: tmp
               mountPath: "/tmp/"
               readOnly: false   
-            {{- if .Values.poller.usernameSecrets }}                 
+            {{- if (.Values.poller).usernameSecrets }}
             - name: snmpv3-secrets
               mountPath: "/app/secrets/snmpv3"
               readOnly: true
@@ -106,7 +108,7 @@ spec:
             items:
               - key: "config.yaml"
                 path: "config.yaml"
-        {{- if .Values.poller.usernameSecrets }}
+        {{- if (.Values.poller).usernameSecrets }}
         - name: snmpv3-secrets
           projected:
             sources:

charts/splunk-connect-for-snmp/templates/traps/deployment.yaml

@@ -48,6 +48,8 @@ spec:
               value: "http://{{ printf "%s-%s" .Release.Name "mibserver" }}/asn1/@mib@"
             - name: MIB_INDEX
               value: "http://{{ printf "%s-%s" .Release.Name "mibserver" }}/index.csv"
+            - name: MIB_STANDARD
+              value: "http://{{ printf "%s-%s" .Release.Name "mibserver" }}/standard.txt"
             - name: LOG_LEVEL
               value: {{ .Values.traps.logLevel | default "INFO" }}
 {{- if .Values.splunk.protocol }}

charts/splunk-connect-for-snmp/templates/worker/deployment.yaml

@@ -58,6 +58,8 @@ spec:
               value: "http://{{ printf "%s-%s" .Release.Name "mibserver" }}/asn1/@mib@"
             - name: MIB_INDEX
               value: "http://{{ printf "%s-%s" .Release.Name "mibserver" }}/index.csv"
+            - name: MIB_STANDARD
+              value: "http://{{ printf "%s-%s" .Release.Name "mibserver" }}/standard.txt"
             {{- if .Values.splunk.enabled }}
             {{- if .Values.splunk.protocol }}
             - name: SPLUNK_HEC_SCHEME
@@ -82,6 +84,14 @@ spec:
                   secretKeyRef:
                     name: {{ include "splunk-connect-for-snmp.name" . }}-splunk
                     key: hec_token
+            {{- if .Values.splunk.eventIndex }}
+            - name: SPLUNK_HEC_INDEX_EVENTS
+              value: {{ .Values.splunk.eventIndex | default "netops" }}
+            {{- end}}
+            {{- if .Values.splunk.metricsIndex }}
+            - name: SPLUNK_HEC_INDEX_METRICS
+              value: {{ .Values.splunk.metricsIndex | default "netmetrics" }}
+            {{- end}}
             {{- end}}
           volumeMounts:
             - name: config
@@ -93,7 +103,7 @@ spec:
             - name: tmp
               mountPath: "/tmp/"
               readOnly: false                    
-            {{- if .Values.poller.usernameSecrets }}                 
+            {{- if (.Values.poller).usernameSecrets }}
             - name: snmpv3-secrets
               mountPath: "/app/secrets/snmpv3"
               readOnly: true
@@ -137,7 +147,7 @@ spec:
             items:
               - key: "config.yaml"
                 path: "config.yaml"
-        {{- if .Values.poller.usernameSecrets }}
+        {{- if (.Values.poller).usernameSecrets }}
         - name: snmpv3-secrets
           projected:
             sources:

docs/configuration/deployment-configuration.md

@@ -30,16 +30,16 @@ All of the components have the `resources` field for adjusting memory resources:
 More information about the concept of `resources` can be found in the [kuberentes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/).
 
 ### Update Inventory and Profile
-Inventory and profiles in `valuses.yaml` is quite expensive from Splunk Connect for SNMP perspective. 
+Inventory and profiles in `values.yaml` is quite expensive from Splunk Connect for SNMP perspective. 
 It required several checks before applying changes. SC4SNMP was designed to prevent changes in inventory and profiles 
 task more often than every 5 min. 
 
-When changing inventory or profile need to be apply in `valuses.yaml` following steps need to be done:
+When changing inventory or profile need to be apply in `values.yaml` following steps need to be done:
 
 1. Apply changes in `values.yaml` 
 2. Check is inventory pod is still running by execute command 
    ```shell script
-   microk8s kubectl -n sc4snmp get pods |grep inventory
+   microk8s kubectl -n sc4snmp get pods | grep inventory
    ```
    If command do not return any pods than follow next step in other case wait and execute command again till moment 
    when inventory job finish. 

docs/configuration/poller-configuration.md

@@ -24,6 +24,8 @@ poller:
     10.202.4.202,,2c,public,,,2000,,,
 ```
 
+NOTE: header's line (`address,port,version,community`) is necessary for the correct execution of SC4SNMP. Do not remove it.
+
 ### Define log level
 Log level for trap can be set by changing value for key `logLevel`. Allowed value are: `DEBUG`, `INFO`, `WARNING`, `ERROR`. 
 Default value is `WARNING`