A new scanner example

[Avamander]

@Avamander If I were to write a new scanner example,

1. I would require that startConstCarrier() take no arguments because the PA Level and RF channel can/should be set independently of startConstCarrier().
2. Furthermore, Any example that involves other peripheral hardware that isn't a nRF24 module (like a TFT display) should be saved to the "examples/recipes/" folder because such an example requires external libraries/hardware that isn't directly supported by RF24 lib.
   • Unfortunately, I don't own a TFT display to test on ☹️ , rather I would probably just use the Arduino IDE's Serial plotter option (which would output slower than an I2C based TFT display).
   • The github action that runs CI with the Arduino CLI compiler could be told to install an external library as well.
3. testRpd() (& testCarrier()) only detect signals "louder" than -64 dBm. The video you linked to is doing some averaging and manipulation on falling peaks. This is what makes your request interesting (to me). Furthermore, this data is only reset when entering RX mode (specifically during FALLING transitions of the CE pin when the CONFIG register's PRIM_RX bit is asserted), so add the confusion of manually manipulating the CE pin (considering that ce() is a private function).

Nonetheless, I shy away from no challenge...

Originally posted by @2bndy5 in #713 (comment)

[Avamander]

I think this is worthy of separate discussion so I split it into a separate issue.

[Avamander]

I've been giving it more thought... Instead of a TFT display, we could use a python program (using the cross-platform freindly pyserial lib for input from the MCU) to output a more visually appealing and/or understandable chart/graph data to a window (probably using pygame or the builtin tkinter libs for drawing). Of course this idea would be even harder for people (like me) ssh-ing to RPi and running examples from the terminal.

[Avamander]

It seems to me that this isn't a trivial example to build. There's Linux machines and Arduino boards that would both need a separate approach.

On Linux it would be possible to build something appealing with C++/ncurses or Python/rich-python without a lot of effort.

On an Arduino board it would theoretically be possible to use Adafruit-GFX API and it should work on a lot of displays.

[2bndy5]

I would require that startConstCarrier() take no arguments because the RF data rate PA Level and RF channel can/should be set independently of startConstCarrier().

After a little testing with existing scanner example, I noticed that if I changed the channel (on a separate nRF24 module) while the Carrier Wave was transmitting, then scanning nRF24 module would show the carrier wave detected on the wrong channel. This needs more investigation, but I have a feeling the module TX-ing the carrier wave isn't modulating to the right frequency while the PLL is locked (because its transmitting)...

Also I haven't tried changing the PA level while transmitting a carrier wave, but that would be harder to verify if the new PA level is achieved correctly. I'm curious if @waltbar did extensive testing before submitting #609 (which is why startConstCarrier() takes args to begin with).

[2bndy5]

Just for reference, it was the comment from @Avamander that started this "new scanner" idea:

@2bndy5 I'm not saying you should do it, but maybe a newer Scanner sketch would still be a nice addition amongst the examples? E.g. with optional TFT support like on this video: https://www.youtube.com/watch?v=kapd3wM9hsg

The youtube video's description has a link to the scanner source code

[Avamander]

but I have a feeling the module TX-ing the carrier wave isn't modulating to the right frequency while the PLL is locked (because its transmitting)...

This sounds like an useful thing to figure out. I wonder if anyone here has an SDR that could answer what piece here is incorrect.

[2bndy5]

On Linux it would be possible to build something appealing with C++/ncurses or Python/rich-python without a lot of effort.

A lot of python people swear by the colorama (3rd-party) lib for changing colors in the terminal. I've used it before as a dependency to a logger lib for python flask (It works rather well).

[Avamander]

A lot of python people swear by the colorama (3rd-party) lib for changing colors in the terminal. I've used it before as a dependency to a logger lib for python flask (It works rather well).

Rich is not just for setting colors, has quite a few nice features/UI-elements built-in that are useful when building a nice quick CLI, I can recommend it.

[2bndy5]

interesting lib (it also requires colorama as a dep), but how do you evision the scanner output?

A quick mock-up from using GIMP

[Avamander]

I was just thinking a progress bar per-channel and just update those with signal strength. (So basically yours but rotated right 90deg)

[2bndy5]

glad I asked. Now I just have to explore the ncursers and rich-python libs A LOT more...

I'm currently tied up in doing a wireless HID hub using circuitpython (something that should mimic Logitech's Unifying dongles)...

[2bndy5]

We need to consider improving the "frame rate" for the scanner output.

I made an attempt at the Arduino+OLED (using I2C) sketch, and the "refresh rate" (using algorithm from original scanner.ino) is drastically slow (about 5 seconds).

[TMRh20]

The timing etc of the scanner example is dictated by the hardware mostly, as the device needs to listen for a bit. Providing the user with a channel range to scan could speed it up instead of scanning the entire range. Or do you mean to display each channel as it is scanned?

…

On Jan 15, 2021, at 7:37 AM, Brendan ***@***.***> wrote:  We need to consider improving the "frame rate" for the scanner output. I made an attempt at the Arduino+OLED (using I2C) sketch, and the "refresh rate" (using algorithm from original scanner.ino) is drastically slow (about 5 seconds). — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[2bndy5]

I was thinking of displaying each channel as it is scanned. Although, providing user with input options (at program start up) isn't a bad idea. Say they want to change the data rate

[2bndy5]

I just tried using \r instead of the default \n in python's print(), and it seems to have a adequate refresh rate (the cursor also looks like a really fast sweeping animated status for the current channel 😃 ) using only the python REPL (no graphic display). Example (using my circuitpython library):

for curr_channel in range(126):  # for each channel
    nrf.channel = curr_channel
    time.sleep(0.00013)  # let the radio finish modulating to the new channel
    nrf.listen = 1  # start a RX session
    time.sleep(0.00013)  # wait 130 microseconds
    signals[curr_channel] += nrf.rpd  # if interference is present
    nrf.listen = 0  # end the RX session

    # output the signal counts per channel
    print(
        hex(min(0x0F, signals[curr_channel]))[2:] if signals[curr_channel] else "-",
        sep="",
        end="" if curr_channel < 125 else "\r",
    )
# print `signals` array 1 last time with a LF at EoL for prettiness

This technique could also be used in the Linux C++ version of the scanner example (using cout instead of printf()).

Last time I tried doing a Serial.print(F("some data \r"));, it didn't work in the Arduino IDE's serial monitor. ☹️ IIRC, it just output \r like a \n.

EDIT: Apparently the "sweeping animated cursor" for the status of the current channel being scanned is specific to using putty. 🤷🏻‍♂️

Question

How should we calculate the decay of the peak for a channel? Should it be time based or the last x number of scans on that channel?

[2bndy5]

@Avamander printing 125 horizontal progress bars may be problematic because each progress bar in rich python is essentially 1 line and the scrolling of terminal windows. Were you thinking of averaging 2 or 3 channels per line?

we can do 3 columns of 41 progress bars or something like that to utilize screen realty. 25 lines seems like a safe bet to avoid automatic scrolling.

[Avamander]

we can do 3 columns of 41 progress bars or something like that to utilize screen realty. 25 lines seems like a safe bet to avoid automatic scrolling.

It'd be possible to read the amount of available lines and divide the channels between those?

[2bndy5]

Python rich has a way of finding the dimensions of the console that we could divy up accordingly.

I still haven't looked at ncurses yet... I imagine they must have an equivalent option.

[2bndy5]

    int  x1 = xx1;
    int  y1 = yy1;
    int  Sc;
    int  x2, y2;
    int  BlobHeight = 4; // Blobs are 4x4 pixels
    char NB[12];         // Number Buffer
    char NB1[12];
    char NB2[12];
    char NB3[12];
    char NB4[12];
    char CB[100]; // COMMAND BUFFER
    char fyll[]   = "fill ";
    char IELLOW[] = "YELLOW";
    char NA[1]    = ""; // blank one

    for (Sc = ScanStart; Sc <= ScanEnd; Sc++) {
        Radio1.setChannel(Sc);
        Radio1.startListening();
        x2 = x1 + (Sc * 5);
        y2 = y1 + 255;
        y2 = y2 - BlobHeight;
        y2 = y2 - AllChannels[Sc];
        delayMicroseconds(120); // Minimum!?
        Radio1.stopListening();
        if (Radio1.testCarrier()) {
            if (AllChannels[Sc] < (250)) {
                AllChannels[Sc] += BlobHeight;
                /* send all channel data to the display */ // EDITED FROM OG SRC
            }
        }
        else {
            NoCarrier[Sc]++;
            if (NoCarrier[Sc] > 15) { // must see no carrier >15 times before reducing the trace
                if (AllChannels[Sc] >= (BlobHeight)) {
                    /* send all channel data to the display */ // EDITED FROM OG SRC
                    AllChannels[Sc] -= (BlobHeight);
                    NoCarrier[Sc] = 0;
                }
            }
        }
    }

[2bndy5]

I noticed there wasn't any python equivalent to scanner.cpp in the examples_linux, so I did some preliminary UI design and rudimentary radio integration with python's rich module.

I have yet to compensate for peak decays, but its working. There seems to be a lot of graphical "page tearing", though that might be due to ssh tunneling.

scanner.py

import time
try:
    from rich.table import Table
    from rich.console import Console
    from rich.progress import BarColumn, Progress, TextColumn
    from rich.live import Live
    from rich.prompt import Prompt, IntPrompt
except ImportError:
    raise OSError(
        "This example requires the python `rich` module installed."
        "\ninstall it using 'python3 -m pip install rich'"
    )
from RF24 import RF24, RF24_1MBPS, RF24_2MBPS, RF24_250KBPS

CSN_PIN = 0  # connected to GPIO8
CE_PIN = 22  # connected to GPIO22
radio = RF24(CE_PIN, CSN_PIN)

if not radio.begin():
    raise RuntimeError("Radio hardware not responding!")
radio.setAutoAck(False)

console = Console()
console.print("1. 1Mbps\n2. 2Mbps\n3. 250kbps")

# create table of progress bars (labeled by frewquency channel in MHz)
table = Table.grid(padding=(0,1))
signals = [0] * 126  # for tracking the signal count on each channel
progress_bars = [Progress()] * 126
for i in range(21):  # 21 rows
    row = []
    for j in range(i, i + (21 * 6), 21):  # 6 columns
        progress_bars[j] = Progress(TextColumn("{task.description}"), BarColumn())
        progress_bars[j].add_task(f"{2400 + (j)}")  # only 1 task for each
        row.append(progress_bars[j])
    table.add_row(*row)

data_rate = int(Prompt.ask("Choose the data rate", choices=["1", "2", "3"], default="1"))
if data_rate == 1:
    radio.setDataRate(RF24_1MBPS)
elif data_rate == 2:
    radio.setDataRate(RF24_2MBPS)
elif data_rate == 3:
    radio.setDataRate(RF24_250KBPS)

timeout = IntPrompt.ask("Enter the scan duration (in whole seconds)")
console.print(
    "Beginning scan using {} for {} seconds. Channel labels are in MHz.".format(
        "1Mbps" if data_rate == 1 else ("2Mbps" if data_rate == 2 else "250kbps"),
        timeout
    )
)


def scan_channel(channel) -> bool:
    """Scan a specified channel and report if a signal was detected."""
    radio.channel = channel
    radio.startListening()
    time.sleep(0.00013)
    result = radio.testRPD()
    radio.stopListening()
    return result


# perform scan
timeout = time.monotonic() + timeout
with Live(table, refresh_per_second=3000) as live:
    try:
        scan_count = 1  # count cycle per entire spectrum
        while time.monotonic() < timeout:
            for i, bar in enumerate(progress_bars):
                signals[i] += scan_channel(i)  # does not handle peak decay
                bar.update(bar.task_ids[0], completed=signals[i], total=scan_count)
            scan_count += 1
    except KeyboardInterrupt:
        radio.powerDown()
        console.print("keyboard interrupt detected; powering down radio.")

[2bndy5]

I just found out that the python std libs ship with a ncurses wrapper (for Linux platforms only). This might be a better alternative to the rich lib (which is written in pure python and a bit sluggish over ssh). Furthermore, the differences between a ncurses C++ scanner example and a python equivalent example would be more manageable.

[2bndy5]

So I put together a couple examples that use the curses lib (1 in python and 1 in C++) on the promiscuous-scanners branch. The python example runs very well, but I have no idea why the C++ example doesn't pick up as many signals (practically nothing). Everything is set up almost identically for both examples...

As for the Arduino example, I have an idea for using macro defines to allow using either an I2C driven display or an SPI driven display. By default, I think I'll demo the I2C driven display since getting 2 SPI devices on 1 bus can be tricky depending on the display being used.

[TMRh20]

I'm getting pretty consistent results between the C++ and Python scanner on my RPI4. My RPi Model B provides much more active output, and I'm not sure the reason, whether it is the radio connected or the RPi version.
At least the RPi4 is consistently picking up signals from my 2 active networks on channels 30 and 50.

One thing to note, if the SSH window is too small, I'm getting an error with the python example:

File "/home/pi/rf24libs/RF24/examples_linux/RF24/examples_linux/scanner.py", line 152, in main
    scanner_output_window = std_scr.subpad(26, curses.COLS, 0, 0)
_curses.error: curses function returned NULL

It may be worth trying to detect the size of the window and giving a non-cryptic error message instead, or at least adding the info to the example.

[2bndy5]

Thanks for testing the C++ curses example, I was starting to think it was my setup somehow (using RPi4 w/ adapter board) I haven't tried the C++ example on my RPi2 (yet).

Both examples don't compensate for dynamically resizing the terminal window; that was something I wanted to add later after proof of concept was established... I'll get to work on that next. TBH, the python example doesn't need to use a separate window (aka "pad"), so I can remove that call completely.

---

I've been prototyping an Arduino scanner with my I2C display, but the detected noise hasn't been ideal. I thought that using my ATSAMD21 board might help with the display refresh rate, but I think there's a bottleneck at the display throughput (w/ or w/o debugging in Serial monitor). Here's a sample pic with display updating once per cycle of spectrum (caching the last 6 scans of the spectrum for peak math):

So far the Arduino code is local only. I wanted to try out my SPI display and see if that refresh rate was any better.

[2bndy5]

I retract my refresh rate complaints. Turns out I had a rogue Serial.println() still executing. And, I keep forgetting that scanning for noise while everyone is still sleeping is pretty much just my own EM footprint (& possibly from neighbors or passing cars). 🤣

Refresh rate is rather good when I toggle a const carrier wave from another device.. Its kinda fun to watch the peak go up and down.

[TMRh20]

Hey, I just realized that testRPD(); will only catch signals >64dBm, but radio.available(); will catch signals <64dBm, so a call to available could be added as well to increase responsiveness even more!
See https://github.com/nRF24/RF24/blob/promiscuous-scanners/examples_linux/scanner.cpp#L150 - modify to

                if (foundSignal || radio.testRPD() || radio.available()) {

[2bndy5]

I thought any payload received had to be >64dbm. Its an interesting idea though because I think the RX FIFO may be getting filled while the RPD flag isn't asserted, so the RX FIFO doesn't seem to be getting flushed when it should be (thus blocking noise detection). In fact, I called flush_rx() before startListening() during my dev tests, and I got a better response from noise. Using available() might help translate that condition into what should be RPD assertion.

[2bndy5]

@TMRh20 Are you using putty for ssh? I've switched to Windows Terminal (I think it ships with Win11 as a default app) for my SSH needs.

Putty's terminal window is typically very small for every new session (using default config). The curses grid needs a 48-character width just to display the channel's labels and signal counts (that's not even including the actual progress bar which uses a dynamically calculated width).

[TMRh20]

Per the datasheet:
Receiver
X Fast AGC for improved dynamic range
X Integrated channel filters
X 13.5mA at 2Mbps
X -82dBm sensitivity at 2Mbps
X -85dBm sensitivity at 1Mbps
X -94dBm sensitivity at 250kbps

I'll have to check out windows terminal, I am using putty.

[2bndy5]

I'll have to check out windows terminal, I am using putty.

You won't be disappointed. It now comes with builtin WSL and VS Dev prompt support, so now I can have a native Ubuntu bash in 1 tab and another tab to compile using VS compiler... And colors 🏳️‍🌈 !

[2bndy5]

I still get the same result with available() added to the condition (in C++ example). I tried it with a carrier wave on channel 99, and it detects the wave sporadically. I really feel like the C++ example is probing the SPI bus too fast or something. I've also tried lowering the SPI speed (my RPi4 setup usually responds better when I do).

Technically, we could save ourselves the extra call to available() if we expose the lib's internal status byte. This is because the internal status byte is updated when we call testRPD(). Its not a great user friendly tactic because the status byte can be outdated (though not in this specific case).

[2bndy5]

I get expected results when I delay a millisecond after setting the new channel. 🤷🏼

[TMRh20]

Yeah, somethings up with it. There is definitely some differences between Arduino behavior and RPi/Linux behavior that I can't explain yet.

[2bndy5]

It doesn't work when I use delayMicroseconds(1000) (or less), but it does work when I use delay(1). I'm going off of memory here, but doesn't delayMicroseconds() use the chrono lib and the delay() uses sys/time.h?

The python curses example works fine, but there's always a slower execution due to the necessary interpreter. And I'm using the python std time lib to delay.

[TMRh20]

I'm pretty sure they both use sys/time.h... they just use different math. Chrono is used for millis(), as other options seem to give messed up results. I'm getting good results here without extra delays.

[2bndy5]

oh yeah, that's right.

[TMRh20]

I've also added some extra addresses to your example, I wasn't sure what to use, but it seems to make a difference:
const uint8_t noiseAddress[][6] = {{0x55, 0x55}, {0xAA, 0xAA}, {0x0A, 0xAA}, {0xA0, 0xAA}, {0x00,0xAA}, {0xAB,0xAA}};

[2bndy5]

I tried it (using 0xA0, 0x0A, 0x50, 0x05), but didn't seem to help any for me. My RPi4 has always seemed to suffer from timing issues (the man_ack example hasn't been 100% reliable for me). I frequently refer to my setup as the culprit, so if it is working for you, then I'm ok with it.

Feel free to commit anything you want to the branch, I'm not going to do any forced git pushes or rebases while testing the branch with my RPis -- that git magic badly messes with my local clones.

[TMRh20]

With further testing, I AM finding a lot more consistent results on my RPi4 with a delay(1); after changing the channel. I'm thinking this is similar to the problems people have had with channel hopping strategies, in that the radio seems to need some additional time to 'settle' after changing the channel, or it rejects packets and does strange things. I don't know really, just a guess as to what is going on here.
With Delay 1:

Without Delay 1

[2bndy5]

My experiences concur, but they aren't entirely specific to the RPi4 (seeing the same for my RPi2 also). I'm temped to try a different driver (been testing with SPIDEV).

radio seems to need some additional time to 'settle' after changing the channel

I suspect that "spamming" the SPI bus too rapidly is the problem. It would make sense if a 100+ microsecond delay after setting the channel solves the inconsistencies, but an entire millisecond makes me think its the SPI bus itself.

[2bndy5]

BTW, I have the Arduino example almost ready to commit. The available() addition proved helpful on that platform also.

[2bndy5]

Ok, I pushed the new Arduino "ScannerGraphic" to the branch. I left some display specific info in the commit message for aaabe5d Its a rough draft right now, but I put it up there for testing and feedback.

I also made some updates to the pico and arduino (serial) scanners that conform to @TMRh20 suggestions. I removed the call to subpad() in the python scanner (untested as of right now).

[2bndy5]

FYI, I'd like to have the filled portion of the bar in C++ scanner use the double horizontal line (═) for better distinction, but curses handles multi-byte chars differently than the curses wrapper does in python. I think I'll resign to using a > symbol instead...

[2bndy5]

Just to add more to the confusion, I tried the C++ curses scanner with the RPi driver, and I got expected results without adding any additional delay after switching channels.

I did notice the example runs a bit slower.

• With SPIDEV, I get about 450+ passes on the entire spectrum after a 25 second scan.
• With bmc2835, I get about 380+ passes on the entire spectrum after a 25 second scan.

[2bndy5]

Looks like the bcm2835 uses register reads to get the system clock if the arg to delayMicroseconds is lower than 450. This requires root privilege though. The comments note that calling nanosleep() (for periods longer than 450 us) takes at least 100 microseconds. This comment might be outdated though (not representative of newer RPi models).

[TMRh20]

Well at least that might indicate why we need a delay with the SPIDEV driver. Originally the SPIDEV drive was much slower, until we figured out automatic CS toggling, and I think there have been some improvements over time. Now its too fast! lol