Skip to content

Commit 57a75fe

Browse files
committed
updates readme
1 parent aa49815 commit 57a75fe

File tree

1 file changed

+47
-0
lines changed

1 file changed

+47
-0
lines changed

README.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# Deploying Spark on Kubernetes
2+
3+
## Want to learn how to build this?
4+
5+
Check out the [post](https://testdriven.io/deploying-spark-on-kubernetes).
6+
7+
## Want to use this project?
8+
9+
### Minikube Setup
10+
11+
Install and run [Minikube](https://kubernetes.io/docs/setup/minikube/):
12+
13+
1. Install a [Hypervisor](https://kubernetes.io/docs/tasks/tools/install-minikube/#install-a-hypervisor) (like [VirtualBox](https://www.virtualbox.org/wiki/Downloads) or [HyperKit](https://github.com/moby/hyperkit)) to manage virtual machines
14+
1. Install and Set Up [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) to deploy and manage apps on Kubernetes
15+
1. Install [Minikube](https://github.com/kubernetes/minikube/releases)
16+
17+
Start the cluster:
18+
19+
```sh
20+
$ minikube start --memory 8192 --cpus 4
21+
$ minikube dashboard
22+
```
23+
24+
Build the Docker image:
25+
26+
```sh
27+
$ eval $(minikube docker-env)
28+
$ docker build -t spark-hadoop:2.2.1 .
29+
```
30+
31+
Create the deployments and services:
32+
33+
```sh
34+
$ kubectl create -f ./kubernetes/spark-master-deployment.yaml
35+
$ kubectl create -f ./kubernetes/spark-master-service.yaml
36+
$ kubectl create -f ./kubernetes/spark-worker-deployment.yaml
37+
$ minikube addons enable ingress
38+
$ kubectl apply -f ./kubernetes/minikube-ingress.yaml
39+
```
40+
41+
Add an entry to /etc/hosts:
42+
43+
```sh
44+
$ echo "$(minikube ip) spark-kubernetes" | sudo tee -a /etc/hosts
45+
```
46+
47+
Test it out in the browser at [http://spark-kubernetes/](http://spark-kubernetes/).

0 commit comments

Comments
 (0)