Skip to content

Upgrading Opensearch to 2.9 #4384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

Upgrading Opensearch to 2.9 #4384

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e8411f1
test
May 1, 2025
df1c5f1
testing local connection to Opensearch
May 1, 2025
a4025ca
Fix OpenSearch issues
aguest-kc May 29, 2025
2c6da1b
Disable the OpenSearch Dashboards security
aguest-kc Aug 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 12 additions & 10 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ services:
- usaspending # must pass --profile usaspending to docker compose for this to come up
- test
- ci
image: docker.elastic.co/elasticsearch/elasticsearch:7.1.1
image: opensearchproject/opensearch:2.9.0
container_name: usaspending-es
environment:
- node.name=usaspending-es
Expand All @@ -165,35 +165,36 @@ services:
- network.host=0.0.0.0
- transport.host=localhost
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms2048m -Xmx2048m" # Ensure Docker is allocated plenty of memory, otherwise this will fail
- plugins.security.disabled=true
- "OPENSEARCH_JAVA_OPTS=-Xms6048m -Xmx6048m" # Ensure Docker is allocated plenty of memory, otherwise this will fail
# Inject plugin install, then resume with orignial entrypoint command
command: >
/bin/sh -c "
if [ ! -d /usr/share/elasticsearch/plugins/mapper-murmur3 ]; then
# Certificate problem workaround when on VPN - wget without checking cert, then install from local filesystem
wget --no-check-certificate https://artifacts.elastic.co/downloads/elasticsearch-plugins/mapper-murmur3/mapper-murmur3-7.1.1.zip
./bin/elasticsearch-plugin install file:///usr/share/elasticsearch/mapper-murmur3-7.1.1.zip
if [ ! -d /usr/share/opensearch/plugins/mapper-murmur3 ]; then
/usr/share/opensearch/bin/opensearch-plugin install mapper-murmur3
fi
/usr/local/bin/docker-entrypoint.sh"
/usr/share/opensearch/opensearch-docker-entrypoint.sh"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- type: volume
source: local_es_data
target: /usr/share/elasticsearch/data
target: /usr/share/opensearch/data
ports:
- 9200:9200

usaspending-kibana-es:
profiles:
- usaspending # must pass --profile usaspending to docker compose for this to come up
image: docker.elastic.co/kibana/kibana-oss:7.1.1
image: opensearchproject/opensearch-dashboards:2.9.0
container_name: usaspending-kibana-es
# ELASTICSEARCH_HOSTS should match the port for "usaspending-es"; value will need to be updated if using Windows
environment:
- ELASTICSEARCH_HOSTS="http://docker.for.mac.localhost:9200"
- OPENSEARCH_HOSTS="http://usaspending-es:9200"
- DISABLE_SECURITY_DASHBOARDS_PLUGIN=true
- DISABLE_SECURITY_PLUGIN=true
ports:
- 5601:5601

Expand Down Expand Up @@ -335,6 +336,7 @@ services:
# make docker-compose-run profiles="--profile spark" args="--rm -e MINIO_HOST=minio -e JDBC_URL -e COMPONENT_NAME='My Spark Prototype Script' spark-submit \
# --packages org.postgresql:postgresql:42.2.23,io.delta:delta-core_2.12:1.2.1,org.apache.hadoop:hadoop-aws:3.3.1,org.apache.spark:spark-hive_2.12:3.2.1 \
# /project/usaspending_api/etl/tests/path_to_your_spark_prototype_script.py"

spark-submit:
profiles:
- spark # must pass --profile spark to docker compose for this to come up
Expand Down
Loading