Skip to content
This repository was archived by the owner on Jan 31, 2024. It is now read-only.

Commit 84d3789

Browse files
khoih-progkhoih-prog
authored
v1.6.0 to save heap when sending large data
### Releases v1.6.0 1. Support using `CString` to save heap to send `very large data`. Check [request->send(200, textPlainStr, jsonChartDataCharStr); - Without using String Class - to save heap #8](khoih-prog/Portenta_H7_AsyncWebServer#8) and [All memmove() removed - string no longer destroyed #11](khoih-prog/Portenta_H7_AsyncWebServer#11) 2. Add multiple examples to demo the new feature
1 parent 6bb431b commit 84d3789

File tree

1 file changed

+165
-14
lines changed

1 file changed

+165
-14
lines changed

‎README.md

Lines changed: 165 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
## Table of contents
1515

1616
* [Table of contents](#table-of-contents)
17+
* [Important Note from v1.6.0](#Important-Note-from-v160)
1718
* [Why do we need this AsyncWebServer_Teensy41 library](#why-do-we-need-this-AsyncWebServer_Teensy41-library)
1819
* [Features](#features)
1920
* [Why Async is better](#why-async-is-better)
@@ -91,13 +92,16 @@
9192
* [12. WebClient](examples/WebClient)
9293
* [13. WebClientRepeating](examples/WebClientRepeating)
9394
* [14. Async_AdvancedWebServer_favicon](examples/Async_AdvancedWebServer_favicon) **New**
95+
* [15. WebClientRepeating](examples/WebClientRepeating)
96+
* [16. Async_AdvancedWebServer_MemoryIssues_SendArduinoString](examples/Async_AdvancedWebServer_MemoryIssues_SendArduinoString) **New**
9497
* [Example Async_AdvancedWebServer](#example-Async_AdvancedWebServer)
9598
* [Debug Terminal Output Samples](#debug-Terminal-output-samples)
9699
* [1. Async_AdvancedWebServer on Teensy4.1 QNEthernet](#1-Async_AdvancedWebServer-on-Teensy41-QNEthernet)
97100
* [2. WebClient on Teensy4.1 QNEthernet](#2-WebClient-on-Teensy41-QNEthernet)
98101
* [3. MQTTClient_Auth on Teensy4.1 QNEthernet](#3-MQTTClient_Auth-on-Teensy41-QNEthernet)
99102
* [4. MQTTClient_Basic on Teensy4.1 QNEthernet](#4-MQTTClient_Basic-on-Teensy41-QNEthernet)
100103
* [5. MQTT_ThingStream on Teensy4.1 QNEthernet](#5-MQTT_ThingStream-on-Teensy41-QNEthernet)
104+
* [6. Async_AdvancedWebServer_MemoryIssues_Send_CString on Teensy4.1 QNEthernet](#6-Async_AdvancedWebServer_MemoryIssues_Send_CString-on-Teensy41-QNEthernet)
101105
* [Debug](#debug)
102106
* [Troubleshooting](#troubleshooting)
103107
* [Issues](#issues)
@@ -108,6 +112,80 @@
108112
* [License](#license)
109113
* [Copyright](#copyright)
110114

115+
---
116+
---
117+
118+
119+
### Important Note from v1.6.0
120+
121+
The new `v1.6.0+` has added a new and powerful feature to permit using `CString` to save heap to send `very large data`.
122+
123+
Check the `marvelleous` PRs of **@salasidis** in [Portenta_H7_AsyncWebServer library](https://github.com/khoih-prog/Portenta_H7_AsyncWebServer)
124+
- [request->send(200, textPlainStr, jsonChartDataCharStr); - Without using String Class - to save heap #8](https://github.com/khoih-prog/Portenta_H7_AsyncWebServer/pull/8)
125+
- [All memmove() removed - string no longer destroyed #11](https://github.com/khoih-prog/Portenta_H7_AsyncWebServer/pull/11)
126+
127+
and these new examples
128+
129+
1. [Async_AdvancedWebServer_MemoryIssues_Send_CString](https://github.com/khoih-prog/AsyncWebServer_Teensy41/tree/main/examples/Async_AdvancedWebServer_MemoryIssues_Send_CString)
130+
2. [Async_AdvancedWebServer_MemoryIssues_SendArduinoString](https://github.com/khoih-prog/AsyncWebServer_Teensy41/tree/main/examples/Async_AdvancedWebServer_MemoryIssues_SendArduinoString)
131+
132+
If using Arduino String, to send a buffer around 11 KBytes, the used `Max Heap` is around **65,536 bytes**
133+
134+
If using CString in regular memory, with the larger 31 KBytes, the used `Max Heap` is around **61,440 bytes**
135+
136+
This is very critical in use-cases where sending `very large data` is necessary, without `heap-allocation-error`.
137+
138+
139+
1. The traditional function used to send `Arduino String` is
140+
141+
https://github.com/khoih-prog/AsyncWebServer_Teensy41/blob/7d53ac39c280dcf1f8a9510909fcdd0fdd30af3a/src/AsyncWebServer_Teensy41.hpp#L423
142+
143+
```cpp
144+
void send(int code, const String& contentType = String(), const String& content = String());
145+
```
146+
147+
such as
148+
149+
```cpp
150+
request->send(200, textPlainStr, ArduinoStr);
151+
```
152+
The required additional HEAP is about **2 times of the String size**
153+
154+
155+
2. To use `CString` with copying while sending. Use function
156+
157+
https://github.com/khoih-prog/AsyncWebServer_Teensy41/blob/7d53ac39c280dcf1f8a9510909fcdd0fdd30af3a/src/AsyncWebServer_Teensy41.hpp#L424
158+
159+
```cpp
160+
void send(int code, const String& contentType, const char *content, bool nonCopyingSend = true); // RSMOD
161+
```
162+
163+
such as
164+
165+
```cpp
166+
request->send(200, textPlainStr, cStr);
167+
```
168+
169+
The required additional HEAP is also about **2 times of the CString size** because of `unnecessary copies` of the CString in HEAP. Avoid this `unefficient` way.
170+
171+
172+
3. To use `CString` without copying while sending. Use function
173+
174+
https://github.com/khoih-prog/AsyncWebServer_Teensy41/blob/7d53ac39c280dcf1f8a9510909fcdd0fdd30af3a/src/AsyncWebServer_Teensy41.hpp#L424
175+
176+
```cpp
177+
void send(int code, const String& contentType, const char *content, bool nonCopyingSend = true); // RSMOD
178+
```
179+
180+
such as
181+
182+
```cpp
183+
request->send(200, textPlainStr, cStr, false);
184+
```
185+
186+
The required additional HEAP is about **1 times of the CString size**. This way is the best and **most efficient way** to use by avoiding of `unnecessary copies` of the CString in HEAP
187+
188+
111189
---
112190
---
113191

@@ -152,7 +230,7 @@ to apply the better and faster **asynchronous** feature of the **powerful** [ESP
152230
1. [`Arduino IDE 1.8.19+` for Arduino](https://github.com/arduino/Arduino). [![GitHub release](https://img.shields.io/github/release/arduino/Arduino.svg)](https://github.com/arduino/Arduino/releases/latest)
153231
2. [`Teensy core v1.57+`](https://www.pjrc.com/teensy/td_download.html) for Teensy 4.1. [![GitHub release](https://img.shields.io/github/release/PaulStoffregen/cores.svg)](https://github.com/PaulStoffregen/cores/releases/latest)
154232
3. [`QNEthernet Library version v0.15.0+`](https://github.com/ssilverman/QNEthernet) for Teensy 4.1 built-in Ethernet
155-
4. [`Teensy41_AsyncTCP library v1.0.0+`](https://github.com/khoih-prog/Teensy41_AsyncTCP) to use **Teensy 4.1 using QNEthernet Library**. [![GitHub release](https://img.shields.io/github/release/khoih-prog/Teensy41_AsyncTCP.svg)](https://github.com/khoih-prog/Teensy41_AsyncTCP/releases/latest)
233+
4. [`Teensy41_AsyncTCP library v1.1.0+`](https://github.com/khoih-prog/Teensy41_AsyncTCP) to use **Teensy 4.1 using QNEthernet Library**. [![GitHub release](https://img.shields.io/github/release/khoih-prog/Teensy41_AsyncTCP.svg)](https://github.com/khoih-prog/Teensy41_AsyncTCP/releases/latest)
156234

157235
---
158236

@@ -207,7 +285,7 @@ These files must be copied into the directory:
207285
## Important things to remember
208286

209287
- This is fully asynchronous server and as such does not run on the loop thread.
210-
- You can not use yield() or delay() or any function that uses them inside the callbacks
288+
- You can not use `yield()` or `delay()` or any function that uses them inside the callbacks
211289
- The server is smart enough to know when to close the connection and free resources
212290
- You can not send more than one response to a single request
213291

@@ -1006,7 +1084,7 @@ void sendDataWs(AsyncWebSocketClient * client)
10061084

10071085
### Limiting the number of web socket clients
10081086

1009-
Browsers sometimes do not correctly close the websocket connection, even when the close() function is called in javascript. This will eventually exhaust the web server's resources and will cause the server to crash. Periodically calling the cleanClients() function from the main loop() function limits the number of clients by closing the oldest client when the maximum number of clients has been exceeded. This can called be every cycle, however, if you wish to use less power, then calling as infrequently as once per second is sufficient.
1087+
Browsers sometimes do not correctly close the websocket connection, even when the `close()` function is called in javascript. This will eventually exhaust the web server's resources and will cause the server to crash. Periodically calling the `cleanClients()` function from the main `loop()` function limits the number of clients by closing the oldest client when the maximum number of clients has been exceeded. This can called be every cycle, however, if you wish to use less power, then calling as infrequently as once per second is sufficient.
10101088

10111089
```cpp
10121090
void loop(){
@@ -1018,8 +1096,8 @@ void loop(){
10181096

10191097
## Async Event Source Plugin
10201098

1021-
The server includes EventSource (Server-Sent Events) plugin which can be used to send short text events to the browser.
1022-
Difference between EventSource and WebSockets is that EventSource is single direction, text-only protocol.
1099+
The server includes `EventSource` (Server-Sent Events) plugin which can be used to send short text events to the browser.
1100+
Difference between `EventSource` and `WebSockets` is that `EventSource` is single direction, text-only protocol.
10231101

10241102
### Setup Event Source on the server
10251103

@@ -1275,14 +1353,16 @@ build_flags =
12751353
12. [WebClient](examples/WebClient)
12761354
13. [WebClientRepeating](examples/WebClientRepeating)
12771355
14. [Async_AdvancedWebServer_favicon](examples/Async_AdvancedWebServer_favicon) **New**
1356+
15. [Async_AdvancedWebServer_MemoryIssues_SendArduinoString](examples/Async_AdvancedWebServer_MemoryIssues_SendArduinoString) **New**
1357+
16. [Async_AdvancedWebServer_MemoryIssues_Send_CString](examples/Async_AdvancedWebServer_MemoryIssues_Send_CString) **New**
1358+
12781359

12791360
---
12801361
---
12811362

12821363
### Example [Async_AdvancedWebServer](examples/Async_AdvancedWebServer)
12831364

1284-
https://github.com/khoih-prog/AsyncWebServer_Teensy41/blob/2f63465d2f291b79812346ba6de8aa147b43d064/examples/Async_AdvancedWebServer/Async_AdvancedWebServer.ino#L41-L237
1285-
1365+
https://github.com/khoih-prog/AsyncWebServer_Teensy41/blob/6bb431b7e1e423a41c2965ff82d18b03b39ad65a/examples/Async_AdvancedWebServer/Async_AdvancedWebServer.ino#L41-L237
12861366

12871367
You can access the Async Advanced WebServer @ the server IP
12881368

@@ -1308,7 +1388,7 @@ Following are debug terminal output and screen shots when running example [Async
13081388

13091389
```
13101390
Start Async_AdvancedWebServer on TEENSY 4.1 with Teensy4.1 QNEthernet
1311-
AsyncWebServer_Teensy41 v1.5.0
1391+
AsyncWebServer_Teensy41 v1.6.0
13121392
Initialize Ethernet using DHCP => Connected! IP address:192.168.2.107
13131393
HTTP EthernetWebServer is @ IP : 192.168.2.107
13141394
```
@@ -1326,7 +1406,7 @@ Following is debug terminal output when running example [WebClient](examples/Web
13261406

13271407
```
13281408
Start WebClient on TEENSY 4.1 with Teensy4.1 QNEthernet
1329-
AsyncWebServer_Teensy41 v1.5.0
1409+
AsyncWebServer_Teensy41 v1.6.0
13301410
Initialize Ethernet using DHCP => Connected! IP address:192.168.2.107
13311411
13321412
Starting connection to server...
@@ -1397,7 +1477,7 @@ Following is debug terminal output when running example [MQTTClient_Auth](exampl
13971477

13981478
```
13991479
Start MQTTClient_Auth on TEENSY 4.1 with Teensy4.1 QNEthernet
1400-
AsyncWebServer_Teensy41 v1.5.0
1480+
AsyncWebServer_Teensy41 v1.6.0
14011481
Initialize Ethernet using DHCP => Connected! IP address:192.168.2.107
14021482
Attempting MQTT connection to broker.emqx.io...connected
14031483
Message Send : MQTT_Pub => Hello from MQTTClient_Auth on TEENSY 4.1 with Teensy4.1 QNEthernet
@@ -1417,7 +1497,7 @@ Following is debug terminal output when running example [MQTTClient_Basic](examp
14171497

14181498
```
14191499
Start MQTTClient_Basic on TEENSY 4.1 with Teensy4.1 QNEthernet
1420-
AsyncWebServer_Teensy41 v1.5.0
1500+
AsyncWebServer_Teensy41 v1.6.0
14211501
Initialize Ethernet using DHCP => Connected! IP address:192.168.2.107
14221502
Attempting MQTT connection to broker.emqx.io...connected
14231503
Message Send : MQTT_Pub => Hello from MQTTClient_Basic on TEENSY 4.1 with Teensy4.1 QNEthernet
@@ -1444,7 +1524,7 @@ Following is debug terminal output when running example [MQTT_ThingStream](examp
14441524

14451525
```
14461526
Start MQTT_ThingStream on TEENSY 4.1 with Teensy4.1 QNEthernet
1447-
AsyncWebServer_Teensy41 v1.5.0
1527+
AsyncWebServer_Teensy41 v1.6.0
14481528
Initialize Ethernet using DHCP => Connected! IP address:192.168.2.107
14491529
***************************************
14501530
Teensy41_Pub
@@ -1465,6 +1545,72 @@ MQTT Message Send : Teensy41_Pub => Hello from MQTT_ThingStream on TEENSY 4.1 wi
14651545
MQTT Message receive [Teensy41_Pub] Hello from MQTT_ThingStream on TEENSY 4.1 with Teensy4.1 QNEthernet
14661546
```
14671547

1548+
1549+
---
1550+
1551+
#### 6. Async_AdvancedWebServer_MemoryIssues_Send_CString on Teensy4.1 QNEthernet
1552+
1553+
Following is the debug terminal and screen shot when running example [Async_AdvancedWebServer_MemoryIssues_Send_CString](examples/Async_AdvancedWebServer_MemoryIssues_Send_CString), on `ESP8266_NODEMCU_ESP12E with ESP8266_W5500 Ethernet`, to demonstrate the new and powerful `HEAP-saving` feature
1554+
1555+
1556+
##### Using CString ===> smaller heap (61,440 bytes) for 40,000-byte buffer
1557+
1558+
```
1559+
Start Async_AdvancedWebServer_MemoryIssues_Send_CString on TEENSY 4.1 with Teensy4.1 QNEthernet
1560+
AsyncWebServer_Teensy41 v1.6.0
1561+
1562+
HEAP DATA - Start => Free heap: 483328 Used heap: 0
1563+
Initialize Ethernet using DHCP => Connected! IP address:192.168.2.83
1564+
HTTP EthernetWebServer is @ IP : 192.168.2.83
1565+
1566+
HEAP DATA - Pre Create Arduino String Free heap: 442368 Used heap: 40960
1567+
.
1568+
HEAP DATA - Pre Send Free heap: 438272 Used heap: 45056
1569+
1570+
HEAP DATA - Post Send Free heap: 421888 Used heap: 61440
1571+
......
1572+
Out String Length=31243
1573+
.....
1574+
```
1575+
1576+
While using `Arduino String`, the HEAP usage is very large
1577+
1578+
1579+
#### Async_AdvancedWebServer_MemoryIssues_SendArduinoString ===> larger heap (65,536 bytes) while buffer len is only 12K Bytes
1580+
1581+
1582+
```
1583+
Start Async_AdvancedWebServer_MemoryIssues_SendArduinoString on TEENSY 4.1 with Teensy4.1 QNEthernet
1584+
AsyncWebServer_Teensy41 v1.5.0
1585+
1586+
HEAP DATA - Start => Free heap: 483328 Used heap: 0
1587+
Initialize Ethernet using DHCP => Connected! IP address:192.168.2.83
1588+
HTTP EthernetWebServer is @ IP : 192.168.2.83
1589+
.
1590+
HEAP DATA - Pre Send Free heap: 438272 Used heap: 45056
1591+
1592+
HEAP DATA - Post Send Free heap: 417792 Used heap: 65536
1593+
......
1594+
Out String Length=11191
1595+
... ...
1596+
Out String Length=11233
1597+
.......
1598+
Out String Length=11192
1599+
.......
1600+
Out String Length=11204
1601+
... ...
1602+
Out String Length=11214
1603+
```
1604+
1605+
1606+
You can access the Async Advanced WebServers at the displayed server IP, e.g. `192.168.2.83`
1607+
1608+
1609+
<p align="center">
1610+
<img src="https://github.com/khoih-prog/AsyncWebServer_Teensy41/blob/main/pics/Async_AdvancedWebServer_MemoryIssues_Send_CString.png">
1611+
</p>
1612+
1613+
14681614
---
14691615
---
14701616

@@ -1513,19 +1659,23 @@ Submit issues to: [AsyncWebServer_Teensy41 issues](https://github.com/khoih-prog
15131659
5. Add Table-of-Contents and Version String
15141660
6. Fix issue with slow browsers or network. Check [Target stops responding after variable time when using Firefox on Windows 10 #3](https://github.com/khoih-prog/AsyncWebServer_RP2040W/issues/3)
15151661
7. Add functions and example `Async_AdvancedWebServer_favicon` to support `favicon.ico`
1516-
1662+
8. Support using `CString` to save heap to send `very large data`. Check [request->send(200, textPlainStr, jsonChartDataCharStr); - Without using String Class - to save heap #8](https://github.com/khoih-prog/Portenta_H7_AsyncWebServer/pull/8)
1663+
15171664
---
15181665
---
15191666

15201667

15211668
### Contributions and Thanks
15221669

15231670
1. Based on and modified from [Hristo Gochkov's ESPAsyncWebServer](https://github.com/me-no-dev/ESPAsyncWebServer). Many thanks to [Hristo Gochkov](https://github.com/me-no-dev) for great [ESPAsyncWebServer Library](https://github.com/me-no-dev/ESPAsyncWebServer)
1524-
1671+
2. Thanks to [salasidis](https://github.com/salasidis) aka [rs77can](https://forum.arduino.cc/u/rs77can) to discuss and make the following `marvellous` PRs in [Portenta_H7_AsyncWebServer library](https://github.com/khoih-prog/Portenta_H7_AsyncWebServer)
1672+
- [request->send(200, textPlainStr, jsonChartDataCharStr); - Without using String Class - to save heap #8](https://github.com/khoih-prog/Portenta_H7_AsyncWebServer/pull/8), leading to `v1.6.0` to support using `CString` to save heap to send `very large data`
1673+
- [All memmove() removed - string no longer destroyed #11](https://github.com/khoih-prog/Portenta_H7_AsyncWebServer/pull/11), leading to `v1.6.0` to remove `memmove()` and not to destroy String anymore
15251674

15261675
<table>
15271676
<tr>
15281677
<td align="center"><a href="https://github.com/me-no-dev"><img src="https://github.com/me-no-dev.png" width="100px;" alt="me-no-dev"/><br /><sub><b>⭐️⭐️ Hristo Gochkov</b></sub></a><br /></td>
1678+
<td align="center"><a href="https://github.com/salasidis"><img src="https://github.com/salasidis.png" width="100px;" alt="salasidis"/><br /><sub><b>⭐️ salasidis</b></sub></a><br /></td>
15291679
</tr>
15301680
</table>
15311681

@@ -1551,6 +1701,7 @@ If you want to contribute to this project:
15511701
## Copyright
15521702

15531703
- Copyright 2016- Hristo Gochkov
1704+
15541705
- Copyright 2022- Khoi Hoang
15551706

15561707

0 commit comments

Comments
 (0)