ESP32 support and easy import as library

[gzahl]

I was working in parallel to @loopsmark on ESP32 support. The code changes are very similar, but i still post my work, since it differs in two import points:

• I use ESPTrueRandom for ESP8266 and ESP32, which simplifies the library dependencies.
• It is possible to import SensESP as a library in a project without any other dependencies.

As an example you can look at https://github.com/gzahl/SensGPS, which compiles for ESP8226 and ESP32 from the same sources. Both only depend explicitly on https://github.com/gzahl/SensESP#esp32

Please feel free to pull either my work or @loopsmark work. But it would be really nice to be able to simply import the upstream version of SensESP into new projects.

On a sidenode:

• The dependency system of platformio is a mess (or i still don't understand it). It seems like all large libraries don't explicitly state their dependencies, but let just scan for them.
• Does anyone know, why the firmware (from the same code) is a lot larger for ESP32? It does not fit into the default partition layout of my 4M chip. The ESP8226 firmware is smaller. Maybe this can be optimized?

[gzahl]

One addition:
My SensGPS setup works fine on ESP8226.
It does also upload and connect to wifi on ESP32, but it does not publish the data correctly. It was already given access to the SignalK eerver on OpenPlotter, but since it didn't publish any data, i removed it there, so i could try a reconnect/reauthorize. But it doesn't come up anymore.
How do i wipe my esp32 so i can reconfigure it?

btw.: Can it be configured from the main.cpp, so i don't have to login, set wifi etc.?

[ba58smith]

One addition:
My SensGPS setup works fine on ESP8226.
It does also upload and connect to wifi on ESP32, but it does not publish the data correctly. It was already given access to the SignalK eerver on OpenPlotter, but since it didn't publish any data, i removed it there, so i could try a reconnect/reauthorize. But it doesn't come up anymore.
How do i wipe my esp32 so i can reconfigure it?

In PlatformIO, when you have the PlatformIO menu up (click the alien head), there is an option to "Erase Flash". That erases the program, but not the wifi credentials. (I think.) In order to erase the wifi credentials, you bring up the ESP by typing its IP address into a browser, then choose "Reset Device."

btw.: Can it be configured from the main.cpp, so i don't have to login, set wifi etc.?

No, but once the ESP boots up, if it has no wifi credentials, it will broadcast a wifi SSID called "Configure sensesp". Connect to it from your phone or computer, then go to 192.168.4.1 and enter your wifi name and password, and name your device (like "ESP_GPS"). When you save it, and it connects, you shouldn't ever have to do that again.

[ba58smith]

I was working in parallel to @loopsmark on ESP32 support. The code changes are very similar, but i still post my work, since it differs in two import points:

• I use ESPTrueRandom for ESP8266 and ESP32, which simplifies the library dependencies.
• It is possible to import SensESP as a library in a project without any other dependencies.

After your previous PR to add some libraries to library.json (which I closed because I told you that's not how you add libraries in PlatformIO), I found out you were right, and I merged your PR. Then I added the Wire library the same way, and now, a SensESP-based project needs ONLY the SensESP library in its platformio.ini.

So your modifications to library.json in this PR aren't necessary, except possibly the addition you made after Wire.

[loopsmark]

One addition:
My SensGPS setup works fine on ESP8226.
It does also upload and connect to wifi on ESP32, but it does not publish the data correctly. It was already given access to the SignalK eerver on OpenPlotter, but since it didn't publish any data, i removed it there, so i could try a reconnect/reauthorize. But it doesn't come up anymore.
How do i wipe my esp32 so i can reconfigure it?

btw.: Can it be configured from the main.cpp, so i don't have to login, set wifi etc.?

The way I do it is directly in the terminal with this command "pio run -t erase" (the -t is for target)

[gzahl]

Ok, thanks for the answers. I erased my esp32 chip and flashed again.
I can login to wifi and get a token request in the signalk server. I can also confirm that request and get over serial
(D) (get_delta)(C1) SKDelta::get_delta: {"updates":[{"source":{"label":"espgps"},"values":[{"path":"espgps.ipaddr","value":"10.10.10.154"}]}]} (D) (poll_access_request)(C1) Polling SK Server for authentication token (D) (poll_access_request)(C1) COMPLETED (I) (poll_access_request)(C1) Permission granted (D) (get_delta)(C1) SKDelta::get_delta: {"updates":[{"source":{"label":"espgps"},"values":[{"path":"espgps.freemem","value":205768},{"path":"espgps.uptime","value":47.411},{"path":"espgps.systemhz","value":110885}]}]} (D) (get_delta)(C1) SKDelta::get_delta: {"updates":[{"source":{"label":"espgps"},"values":[{"path":"espgps.freemem","value":205948},{"path":"espgps.uptime","value":48.411}]}]}

But i do not get any of the data after this in the signalk server.
Any idea why the request works, but the data publishing not?

So your modifications to library.json in this PR aren't necessary, except possibly the addition you made after Wire.
Ok, i guess my relevant additions there is to specify the "platforms" argument for some of the dependencies.
I will try later to see which additions are really necessary.

[ba58smith]

I can login to wifi and get a token request in the signalk server. I can also confirm that request and get over serial
(D) (get_delta)(C1) SKDelta::get_delta: {"updates":[{"source":{"label":"espgps"},"values":[{"path":"espgps.ipaddr","value":"10.10.10.154"}]}]} (D) (poll_access_request)(C1) Polling SK Server for authentication token (D) (poll_access_request)(C1) COMPLETED (I) (poll_access_request)(C1) Permission granted (D) (get_delta)(C1) SKDelta::get_delta: {"updates":[{"source":{"label":"espgps"},"values":[{"path":"espgps.freemem","value":205768},{"path":"espgps.uptime","value":47.411},{"path":"espgps.systemhz","value":110885}]}]} (D) (get_delta)(C1) SKDelta::get_delta: {"updates":[{"source":{"label":"espgps"},"values":[{"path":"espgps.freemem","value":205948},{"path":"espgps.uptime","value":48.411}]}]}

But i do not get any of the data after this in the signalk server.
Any idea why the request works, but the data publishing not?

In my case, as soon as the the security access request is approved, I immediately start seeing the data in Instrument Panel. When you approved the security request, did you change the permissions from the default "read only" to "read/write"? In the Server UI, go to Security - Devices and you can check the Permissions for each device.

[gzahl]

Yes, i activated security level read & write.
I am trying to debug this now. If i restart the ESP32 it seems SignalK does not even list the connection anymore, e.g. the ESP32 is not getting connected properly. But it shows no interesting output on console either.
I activate debug mode and am trying to figure out what is happening now.
@ba58smith Did you also test with an ESP32? Which one? I have a esp32doit-devkit-v1 board.

[ba58smith]

@gzahl - I have not yet tested THIS Pull Request at all. I did test PR #115 with a Wemos D1 Mini ESP8266, and it compiled and uploaded my four different projects (4 different ESP8266's around my boat), and they're all running that version now. Before I do any testing on this PR, I want to test #115 with at least one of my ESP32's.
Have you tested this PR with an ESP8266 yet? If not, do you have one, and can you test on it?

[gzahl]

Have you tested this PR with an ESP8266 yet? If not, do you have one, and can you test on it?
Yes, as i stated earlier:
My SensGPS setup works fine on ESP8226.

I only have problems with my ESP32. @loopsmark tested #115 with his ESP32 as i understand. I have to check for differencees again.

[gzahl]

I have made one fix to my PR and now it works fine with my ESP32 and my ESP8266.

[gzahl]

There still seems to be a problem. Sometimes it works without problems, but sometimes and then after every reboot my esp32 crashes directly after connecting to the wifi. It still manages to get some data through to signalk.
Here comes the Stacktrace. Any ideas?
`
rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:8896
load:0x40080400,len:5828
entry 0x400806ac
Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.
Core 1 register dump:
PC : 0x401b7402 PS : 0x00060830 A0 : 0x801691fa A1 : 0x3ffb1ea0
A2 : 0x3ffb2c00 A3 : 0x3ffb2c24 A4 : 0x00000050 A5 : 0x3ffb1dc0
A6 : 0x00000000 A7 : 0x00000000 A8 : 0x8016dafe A9 : 0x3ffb1e80
A10 : 0xfefefefe A11 : 0x00000000 A12 : 0xabb40333 A13 : 0x3ffc1948
A14 : 0x3ffd79e0 A15 : 0x3ffb1f70 SAR : 0x0000000a EXCCAUSE: 0x0000001c
EXCVADDR: 0xfefefefe LBEG : 0x4000c2e0 LEND : 0x4000c2f6 LCOUNT : 0xffffffff

Backtrace: 0x401b7402:0x3ffb1ea0 0x401691f7:0x3ffb1ec0 0x400d96e3:0x3ffb1ee0 0x400d1fab:0x3ffb1f00 0x400d21c0:0x3ffb1f20 0x400d20bd:0x3ffb1f50 0x400d20dd:0x3ffb1f70 0x400d20ed:0x3ffb1f90 0x400de009:0x3ffb1fb0 0x40088f69:0x3ffb1fd0
#0 0x401b7402:0x3ffb1ea0 in WebSocketsClient::clientIsConnected(WSclient_t*) at .pio/libdeps/esp32doit-devkit-v1/WebSockets_ID549/src/WebSocketsClient.cpp:461
#1 0x401691f7:0x3ffb1ec0 in WebSocketsClient::loop() at /home/mjung/.platformio/packages/framework-arduinoespressif32/cores/esp32/WString.h:85
#2 0x400d96e3:0x3ffb1ee0 in WSClient::loop() at .pio/libdeps/esp32doit-devkit-v1/SensESP/src/net/ws_client.cpp:383
(inlined by) operator() at .pio/libdeps/esp32doit-devkit-v1/SensESP/src/net/ws_client.cpp:56
(inlined by) _M_invoke at /home/mjung/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1871
#3 0x400d1fab:0x3ffb1f00 in std::function<void ()>::operator()() const at /home/mjung/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/forward_list.h:805
#4 0x400d21c0:0x3ffb1f20 in RepeatReaction::tick() at .pio/libdeps/esp32doit-devkit-v1/ReactESP_ID2965/src/ReactESP.cpp:39
#5 0x400d20bd:0x3ffb1f50 in ReactESP::tickTimed() at /home/mjung/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/forward_list.h:805
#6 0x400d20dd:0x3ffb1f70 in ReactESP::tick() at /home/mjung/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/forward_list.h:805
#7 0x400d20ed:0x3ffb1f90 in loop() at /home/mjung/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/forward_list.h:805
#8 0x400de009:0x3ffb1fb0 in loopTask(void*) at /home/mjung/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:19
#9 0x40088f69:0x3ffb1fd0 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

Rebooting...`

[ba58smith]

Sorry - I've been at sea for 72 hours, just now seeing this. I should be able to provide a better comment later today.

[ba58smith]

@gzahl - I'm not much good at reading a stack trace, but the second-to-last line says

#8 0x400de009:0x3ffb1fb0 in loopTask(void*) at /home/mjung/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:19

Does that mean that the problem came from line 19 of your main.cpp? If so - what is that line doing?

[gzahl]

Hi, no problem, i had not much time myself, but our sailing vessel is now afloat :).
No, it does not mean that it origins from my code. This is the call stack - the problems arises from line #0. #0 is in a function/method which is called from #1 and so on. Therefore it will always origin from main.
I think it might be a bug in one of our library (if run on esp32 only), because i saw similar problems online. But i need to search a little more.

edit: After updating platformio and some libraries everything worked smooth this evening. I guess i will go on developing and see if the issue comes back. Otherwise it might have been fixed in the update.

[ba58smith]

edit: After updating platformio and some libraries everything worked smooth this evening. I guess i will go on developing and see if the issue comes back. Otherwise it might have been fixed in the update.

Didn't work for me. I'm using the GEEEKNET32 ESP32 v1.04 board that uses the WROOM-32 chip (https://www.seeedstudio.com/GeeekNET-ESP32-Development-Board-p-2945.html). The board I'm using in PlatformIO is esp32dev, which was suggested on the PIO forum for any board not specifically defined that uses the WROOM-32 chip.

[env:esp32dev]
platform = espressif32
board = esp32dev

I was able to get the captive portal, configure my wifi credentials, give the ESP32 a hostname, then it restarted and connected to my wifi, and enabled all sensors and transforms. But then it shows "Initiating connection" (which is at line 115 in ws_client.cpp)

void WSClient::connect() {
  if (connection_state!=disconnected) {
    return;
  }
  debugD("Initiating connection");

  connection_state = connecting;

And then, I get this:

␛[0m(D) (connect)(C1) Initiating connection
␛[0mGuru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
Core 1 register dump:
PC      : 0x4018e84e  PS      : 0x00060030  A0      : 0x8014aaf5  A1      : 0x3ffb1ea0  
A2      : 0x3ffb2820  A3      : 0x3ffb2844  A4      : 0x00000050  A5      : 0x3ffc4804  
A6      : 0x00000000  A7      : 0x00000000  A8      : 0x800df8b6  A9      : 0x3ffb1e80  
A10     : 0xfefefefe  A11     : 0x00060820  A12     : 0x4008b79b  A13     : 0x3ffb1e60  
A14     : 0x00050823  A15     : 0x3ffb8058  SAR     : 0x0000001b  EXCCAUSE: 0x0000001c  
EXCVADDR: 0xfefefefe  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff  

Backtrace: 0x4018e84e:0x3ffb1ea0 0x4014aaf2:0x3ffb1ec0 0x400db878:0x3ffb1ee0 0x400d2ed3:0x3ffb1f00 0x400d30e8:0x3ffb1f20 0x400d2fe5:0x3ffb1f50 0x400d3005:0x3ffb1f70 0x400d3015:0x3ffb1f90 0x400e0315:0x3ffb1fb0 0x40088f69:0x3ffb1fd0

Rebooting...

and then it keeps going through the above over and over.

I have your latest code, so in library.json is this:

{
      "name": "ESP Async WebServer",
      "version": "https://github.com/me-no-dev/ESPAsyncWebServer"
    },

which is what you recently changed that fixed your problem. But it doesn't fix mine. Now what? Any ideas?

[gzahl]

Hi, it does also still crash for me. But it does some crash->reboot->crash->reboot->.. etc and sometimes doesn't crash when initiating the connection and then it works. Of course we have to sort this out...
First you should define
monitor_filters = esp32_exception_decoder
in platformio.ini. If you do that, you get a stacktrace as i posted before. Then we will know if it is crashing at the same line as it does for me.

[ba58smith]

At this page (https://github.com/me-no-dev/EspExceptionDecoder/blob/master/src/EspExceptionDecoder.java), I find that "LoadProhibited" means "A load referenced a page mapped with an attribute that does not permit loads". But that doesn't mean anything to me. I'm still trying to track this down, but I'm pretty sure it has to do with WSClient::connect().

[gzahl]

It seems to work for me, if i don't actually load/use any sensors. So only calling
sensesp_app->enable();
but not setup_gps(..).
Can you try, if that helps you?

[ba58smith]

I finally got past the errors, but I'm not sure what caused them. Initially, my SK Server wasn't actually ready for sensors to connect to it, so while SensESP was waiting for mDNS, it would fail and restart, over and over. When I restarted SK Server, then it got past the mDNS problem, and then it submitted a request for security authorization. Since it was waiting for the request to be approved (on the Server), it kept failing and restarting. When I approved it on the Server, then it worked - started sending data to the Server. I'm using the AnalogInput example (which had to be modified to allow a different input pin, since ESP32 has more than one AnalogIn), and it's reading values on pin 34 (one of the pins that can be used with AnalogRead), and it's working fine.

So - it seems that in at least some cases, when waiting for the SK Server to reply to something, it fails and restarts, but I don't know why.

Tomorrow I will try several other examples, and will connect actual sensors for as many of them as I can.

[ba58smith]

I've been able to compile my three actual programs (the ones I use on my boat), and most of the ones in /examples. (The ones that don't compile are missing files that are specific to the example, and I haven't tried to resolve those.) My three actual programs run on the ESP32 I have (GEEEKNET32 ESP32 v1.04, and use [env:esp32dev] in platformio.ini), and so do most of the /examples I tried, but I don't have sensors for most of them, so don't really know if they work.

One of my actual programs uses 1-Wire temp sensors, and the sensors are NOT recognized. However, running a simple Sketch from the Arduino IDE that's supposed to find the addresses of 1-Wire sensors doesn't work on my ESP32, either. So I can't blame ESP for that.

I want to test my three actual ESP8266's with this code next.

[ba58smith]

1-Wire update: I found this (below), that suggested some ESP32's don't work with the DallasTemperature::getDeviceCount(), always returning 0, but that the sensors are recognized and temps can be retrieved the normal way. So I hardcoded num_sensors in onewire_temperature.cpp to 3 (the number of my sensors), and it works. I will see if anyone has found a solution to this yet, but if not, we could make num_sensors be a parameter to DallasTemperatureSensors' constructor to get around this.

Or maybe take this approach - another way to count sensors that should work with ESP8266 and ESP32: milesburton/Arduino-Temperature-Control-Library#141

[ba58smith]

I found out that a change to the OneWire library (sometime after v 2.3.3) broke the DallasTemperature::getDeviceCount() method for ESP32. So I'll just redo how sensors are registered that won't rely on that method.

[ba58smith]

I've tested this PR with an ESP32 with some actual sensors, connecting to the SK Server and sending data properly (displays in Instrument Panel). I've tested it with my three "installed on the boat" Wemos D1 Mini's, getting data from DigitalInputValue, AnalogInput, OneWire, ADS1115 ADC, and DigitalInputCounter. I see no change in behavior, so I'm going to merge this, and then fix a small number of issues I found with the ESP32, in another PR.