Skip to content

Commit bfd5c09

Browse files
leondavileondavi
committed
Merge branch 'master' of github.com:leondavi/NErlNet
2 parents 1da71c8 + f78ed7d commit bfd5c09

27 files changed

+3719
-226
lines changed

‎README.md‎

Lines changed: 7 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,18 @@
11
![Version](https://img.shields.io/github/v/release/leondavi/NErlNet)
22
![Contributors](https://img.shields.io/github/contributors/leondavi/NErlNet)
33
![Issues](https://img.shields.io/github/issues/leondavi/NErlNet)
4-
<a href="https://discord.com/channels/914616114204516393/914616114833670159">
5-
<img src="https://img.shields.io/discord/914616114204516393?logo=discord"
6-
alt="chat on Discord"></a>
4+
[![Discord](https://shields.microej.com/discord/914616114204516393)](https://discord.gg/xwBTbzER)
5+
[![LinkedIn](https://img.shields.io/badge/Linkedin-%230077B5.svg?logo=linkedin&logoColor=white)](https://www.linkedin.com/company/nerlnet)
6+
[![YouTube](https://img.shields.io/badge/YouTube-%23FF0000.svg?logo=YouTube&logoColor=white)](https://www.youtube.com/channel/UCnnWPPKiHioTBy7Zq5shrQw)
7+
[![Hugging Face](https://img.shields.io/badge/Hugging%20Face-FFD21E?logo=huggingface&logoColor=000)](https://huggingface.co/Nerlnet)
78

89
# NErlNet
910

1011
<p align="center">
1112
<img src="NerlnetLogo.png" width="200" title="NerlNet">
1213
</p>
1314

14-
Nerlnet is an open-source library for researching distributed machine learning algorithms on IoT devices. It provides comprehensive insights into both edge devices that run neural network models and network performance and statistics. Nerlnet can simulate distributed ML clusters on a single or multiple machines and deploy these clusters, with minor changes, on various kinds of IoT devices.
15+
Nerlnet is an open-source framework for research and deployment of distributed machine learning algorithms on IoT devices. It provides comprehensive insights into both edge devices that run neural network models and network performance and statistics. Nerlnet can simulate distributed ML clusters on a single or multiple machines and deploy these clusters, with minor changes, on various kinds of IoT devices.
1516

1617
Nerlnet simplifies the setup of a distributed cluster that consists of many models on its edge, communication flow can be fully controlled and monitored, and Nerlnet's Python API allows users to manage and gather data from the distributed cluster throughout the experiment.
1718

@@ -79,17 +80,12 @@ Number of messages, throughput, loss, predictions, models performance, etc.
7980
1.1 Use -h to see the help menu of NerlnetJupyterLaunch.sh script.
8081
1.2 If --no-venv option is selected then required modules can be read from ```src_py/requirements.txt```.
8182
3. Read the instructions of importing Api-Server within the generated readme.md file inside <experiment_directory> folder.
82-
4. Follow the example: https://github.com/leondavi/NErlNet/blob/master/examples/example_run.ipynb
83+
4. Follow the [Example Notebook](https://github.com/leondavi/NErlNet/blob/master/examples/example_run.ipynb)
8384

84-
Visit our hugging face organization page for more datasets and models of Nerlnet.
85-
[<img width="150" alt="hf-logo-with-title" src="https://github.com/leondavi/NErlNet/assets/18975070/93e736b9-732e-4d33-a51a-ca5f68308772">](https://huggingface.co/Nerlnet)
86-
### Social
85+
### Distributed ML on The Edge
8786
Distributed ML on the edge - A new evolution step of AI.
8887

8988
https://github.com/leondavi/NErlNet/assets/18975070/15a3957a-3fd6-4fb2-a365-7e1578468298
90-
91-
[<img width="20" alt="nerlnet-linkedin-page" src="https://github.com/leondavi/NErlNet/assets/18975070/b39d6793-2c31-4cfd-8436-c495267c353a">](https://www.linkedin.com/company/nerlnet)
92-
9389

9490
## Gratitudes
9591
<h3 align="center">Microsoft Azure</h1>

‎examples/syntetic_norm/14DeviceDistFedExp/14DevicesExp.ipynb‎

Lines changed: 1275 additions & 0 deletions
Large diffs are not rendered by default.

‎examples/syntetic_norm/14DevicesDistributed/14DevicesDistributed.ipynb‎

Lines changed: 1289 additions & 0 deletions
Large diffs are not rendered by default.

‎inputJsonsFiles/ConnectionMap/conn_fed_dist_14d.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"connectionsMap":
3+
{
4+
"r1":["mainServer", "r2", "s1", "c3", "c2"],
5+
"r2":["r3", "s2", "s5", "c1"],
6+
"r3":["r4", "s3", "s4"],
7+
"r4":["r1", "c4","c5"]
8+
}
9+
}

‎inputJsonsFiles/DistributedConfig/dc_AEC_1d_2c_1s_4r_4w.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
"devices": [
1515
{
1616
"name": "pc1",
17-
"ipv4": "10.211.55.3",
17+
"ipv4": "10.0.0.5",
1818
"entities": "c1,c2,r2,r1,r3,r4,s1,apiServer,mainServer"
1919
}
2020
],

‎inputJsonsFiles/DistributedConfig/dc_dist_14d.json‎

Lines changed: 259 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,259 @@
1+
{
2+
"nerlnetSettings": {
3+
"frequency": "100",
4+
"batchSize": "50"
5+
},
6+
"mainServer": {
7+
"port": "8900",
8+
"args": ""
9+
},
10+
"apiServer": {
11+
"port": "8901",
12+
"args": ""
13+
},
14+
"devices": [
15+
{
16+
"name": "c0vm0",
17+
"ipv4": "10.0.0.5",
18+
"entities": "mainServer,apiServer"
19+
},
20+
{
21+
"name": "minionMS",
22+
"ipv4": "10.0.0.31",
23+
"entities": "s1"
24+
},
25+
{
26+
"name": "minion0",
27+
"ipv4": "10.0.0.17",
28+
"entities": "s2"
29+
},
30+
{
31+
"name": "minion1",
32+
"ipv4": "10.0.0.18",
33+
"entities": "s3"
34+
},
35+
{
36+
"name": "minion2",
37+
"ipv4": "10.0.0.19",
38+
"entities": "s4"
39+
},
40+
{
41+
"name": "minion3",
42+
"ipv4": "10.0.0.20",
43+
"entities": "s5"
44+
},
45+
{
46+
"name": "minion4",
47+
"ipv4": "10.0.0.21",
48+
"entities": "c1"
49+
},
50+
{
51+
"name": "minion5",
52+
"ipv4": "10.0.0.22",
53+
"entities": "c2"
54+
},
55+
{
56+
"name": "minion6",
57+
"ipv4": "10.0.0.23",
58+
"entities": "c3"
59+
},
60+
{
61+
"name": "minion7",
62+
"ipv4": "10.0.0.24",
63+
"entities": "c4"
64+
},
65+
{
66+
"name": "minion8",
67+
"ipv4": "10.0.0.25",
68+
"entities": "c5"
69+
},
70+
{
71+
"name": "minion9",
72+
"ipv4": "10.0.0.26",
73+
"entities": "r1"
74+
},
75+
{
76+
"name": "minion10",
77+
"ipv4": "10.0.0.27",
78+
"entities": "r2"
79+
},
80+
{
81+
"name": "minion11",
82+
"ipv4": "10.0.0.28",
83+
"entities": "r3"
84+
},
85+
{
86+
"name": "minion12",
87+
"ipv4": "10.0.0.29",
88+
"entities": "r4"
89+
}
90+
],
91+
"routers": [
92+
{
93+
"name": "r1",
94+
"port": "8900",
95+
"policy": "0"
96+
},
97+
{
98+
"name": "r2",
99+
"port": "8900",
100+
"policy": "0"
101+
},
102+
{
103+
"name": "r3",
104+
"port": "8900",
105+
"policy": "0"
106+
},
107+
{
108+
"name": "r4",
109+
"port": "8900",
110+
"policy": "0"
111+
}
112+
],
113+
"sources": [
114+
{
115+
"name": "s1",
116+
"port": "8900",
117+
"frequency": "50",
118+
"policy": "0",
119+
"epochs": "2",
120+
"type": "0"
121+
},
122+
{
123+
"name": "s2",
124+
"port": "8900",
125+
"frequency": "50",
126+
"policy": "0",
127+
"epochs": "2",
128+
"type": "0"
129+
},
130+
{
131+
"name": "s3",
132+
"port": "8900",
133+
"frequency": "50",
134+
"policy": "0",
135+
"epochs": "2",
136+
"type": "0"
137+
},
138+
{
139+
"name": "s4",
140+
"port": "8900",
141+
"frequency": "50",
142+
"policy": "0",
143+
"epochs": "2",
144+
"type": "0"
145+
},
146+
{
147+
"name": "s5",
148+
"port": "8900",
149+
"frequency": "50",
150+
"policy": "0",
151+
"epochs": "2",
152+
"type": "0"
153+
}
154+
],
155+
"clients": [
156+
{
157+
"name": "c1",
158+
"port": "8900",
159+
"workers": "w1,w2"
160+
},
161+
{
162+
"name": "c2",
163+
"port": "8900",
164+
"workers": "w3,w4"
165+
},
166+
{
167+
"name": "c3",
168+
"port": "8900",
169+
"workers": "w5,w6"
170+
},
171+
{
172+
"name": "c4",
173+
"port": "8900",
174+
"workers": "w7,w8"
175+
},
176+
{
177+
"name": "c5",
178+
"port": "8900",
179+
"workers": "w9,w10"
180+
}
181+
],
182+
"workers": [
183+
{
184+
"name": "w1",
185+
"model_sha": "0771693392e898393c9b2b8235497537b5fbed1fd0c9a5a7ec6aab665d2c1896"
186+
},
187+
{
188+
"name": "w2",
189+
"model_sha": "0771693392e898393c9b2b8235497537b5fbed1fd0c9a5a7ec6aab665d2c1896"
190+
},
191+
{
192+
"name": "w3",
193+
"model_sha": "0771693392e898393c9b2b8235497537b5fbed1fd0c9a5a7ec6aab665d2c1896"
194+
},
195+
{
196+
"name": "w4",
197+
"model_sha": "0771693392e898393c9b2b8235497537b5fbed1fd0c9a5a7ec6aab665d2c1896"
198+
},
199+
{
200+
"name": "w5",
201+
"model_sha": "0771693392e898393c9b2b8235497537b5fbed1fd0c9a5a7ec6aab665d2c1896"
202+
},
203+
{
204+
"name": "w6",
205+
"model_sha": "0771693392e898393c9b2b8235497537b5fbed1fd0c9a5a7ec6aab665d2c1896"
206+
},
207+
{
208+
"name": "w7",
209+
"model_sha": "0771693392e898393c9b2b8235497537b5fbed1fd0c9a5a7ec6aab665d2c1896"
210+
},
211+
{
212+
"name": "w8",
213+
"model_sha": "0771693392e898393c9b2b8235497537b5fbed1fd0c9a5a7ec6aab665d2c1896"
214+
},
215+
{
216+
"name": "w9",
217+
"model_sha": "0771693392e898393c9b2b8235497537b5fbed1fd0c9a5a7ec6aab665d2c1896"
218+
},
219+
{
220+
"name": "w10",
221+
"model_sha": "0771693392e898393c9b2b8235497537b5fbed1fd0c9a5a7ec6aab665d2c1896"
222+
}
223+
],
224+
"model_sha": {
225+
"0771693392e898393c9b2b8235497537b5fbed1fd0c9a5a7ec6aab665d2c1896": {
226+
"modelType": "0",
227+
"_doc_modelType": " nn:0 | approximation:1 | classification:2 | forecasting:3 | image_classification:4 | text_classification:5 | text_generation:6 | auto_association:7 | autoencoder:8 | ae_classifier:9 |",
228+
"modelArgs": "",
229+
"_doc_modelArgs": "Extra arguments to model",
230+
"layersSizes": "5,16,8,3",
231+
"_doc_layersSizes": "List of postive integers [L0, L1, ..., LN]",
232+
"layerTypesList": "1,3,3,3",
233+
"_doc_LayerTypes": " Default:0 | Scaling:1 | Conv:2 | Perceptron:3 | Pooling:4 | Probabilistic:5 | LSTM:6 | Reccurrent:7 | Unscaling:8 | Flatten:9 | Bounding:10 |",
234+
"layers_functions": "1,7,7,11",
235+
"_doc_layers_functions_activation": " Threshold:1 | Sign:2 | Logistic:3 | Tanh:4 | Linear:5 | ReLU:6 | eLU:7 | SeLU:8 | Soft-plus:9 | Soft-sign:10 | Hard-sigmoid:11 |",
236+
"_doc_layer_functions_pooling": " none:1 | Max:2 | Avg:3 |",
237+
"_doc_layer_functions_probabilistic": " Binary:1 | Logistic:2 | Competitive:3 | Softmax:4 |",
238+
"_doc_layer_functions_scaler": " none:1 | MinMax:2 | MeanStd:3 | STD:4 | Log:5 |",
239+
"lossMethod": "2",
240+
"_doc_lossMethod": " SSE:1 | MSE:2 | NSE:3 | MinkowskiE:4 | WSE:5 | CEE:6 |",
241+
"lr": "0.001",
242+
"_doc_lr": "Positve float",
243+
"epochs": "1",
244+
"_doc_epochs": "Positve Integer",
245+
"optimizer": "5",
246+
"_doc_optimizer": " GD:0 | CGD:1 | SGD:2 | QuasiNeuton:3 | LVM:4 | ADAM:5 |",
247+
"optimizerArgs": "none",
248+
"_doc_optimizerArgs": "String",
249+
"infraType": "0",
250+
"_doc_infraType": " opennn:0 | wolfengine:1 |",
251+
"distributedSystemType": "0",
252+
"_doc_distributedSystemType": " none:0 | fedClientAvg:1 | fedServerAvg:2 |",
253+
"distributedSystemArgs": "SyncMaxCount=300",
254+
"_doc_distributedSystemArgs": "String",
255+
"distributedSystemToken": "9922u",
256+
"_doc_distributedSystemToken": "Token that associates distributed group of workers and parameter-server"
257+
}
258+
}
259+
}

0 commit comments

Comments
 (0)