Sunday, 29 January 2023

Complex Network Analysis

 I've recently been working on two fascinating, challenging projects.

One is a guide to the Pimoroni Explorer Base. That's going to take a while, and I'll report on progress as i go.

The other is a tool to help me keep track of published papers on some of the AI areas the interest me.

Therese's so much published that it's a real challenge to keep up. I rely heavily on the wonderful service provided by Semantic Scholar; that helps me keep track of new papers that cite older papers that I've identified as interesting.

Here's the citation  network for one of the papers that interest me:

I've done quite a bit of work on networks with Python, but I've recently been reading Complex Network Analysis in Python by Dmitry Zinoviev. It's well-written, broad in coverage, and has plenty of useful sample code.

I love it. 

Thursday, 25 August 2022

How to write reliable tests for Python MQTT applications

More and more IoT applications use MQTT. It's a simple and very useful messaging framework which runs on small boards like the Raspberry Pi Pico as well as systems running under Linux, MacOS and Windows.

I recently decided to add some extra functionality to Lazydoro using MQTT. The code seemed to work when run manually but I had a lot of trouble getting my automated tests working. It took quite a while to understand the problem, but the fix was simple.

Intermittently failing tests are bad

In the end-to-end test that was causing the problem, the code simulated the start of a pomodoro session and then checked that the correct MQTT message had been  sent. The test usually failed but sometimes passed. When I manually ran a separate client that subscribed to the message stream I could see that the right messages were being sent.

Intermittently failing (or passing) tests are a nuisance. They do nothing to build confidence that the application under test is working reliably, and they are no help when you're refactoring. You can never be sure if the tested fail because you made a mistake in the refactoring, or was it just having one of its hissy fits?

Solving timing problems

Intermittent failures like this are often due to timing issues. It's tempting to solve them by adding delays to the testing code, but this is prone to problems. Too short a delay, and the tests still fail from time to time; too long a delay, and the tests become burdensome to run.

The solution is simple; write your test so that it polls to see if the expected condition is true, and set a  timeout so that it will only expire if the test is going to fail.

Before the test checks that the correct message has been sent, it waits until there is a message to check.

Here's the code that waits:

 def wait_for_message(self, tries = 100, interval = 0.01):
        for i in range(tries):
            if len(self.messages()) > 0:
        raise ValueError('waiting for message - timed out')
Now the tests run reliably.

You can see the entire Test Client code here.

Thursday, 11 August 2022

How can you make your Pico/Pico W project portable?

In this article you'll learn how to solder the Pimoroni Lipo Shim for Raspberry Pi Pico/Pico W and use it to power your project from a single LiPo battery. You'll also discover a problem, and a solution, if you want to check the battery level remotely.

Some Pico W projects need to work anywhere. Wi-Fi takes care of the connectivity, but the projects need battery power to make them truly portable.

If you're building a portable project you'll find the Pimoroni Lipo Shim for Pico is a great solution. Here's how to attach and use it.

Powering the Pico

While you're writing the software for your Pico or Pico are you'll be using a USB cable to linking it to your host. Once your hardware and software are working you may want to free the Pico from its umbilical cord.

The Pico can be powered from a battery; it's very adaptable, working off an input voltage that can range from 1.8 to 5.5 volts.

The Pimoroni Lipo Shim for Pico takes advantage of that. It lets you power your project from a compact LiPo battery, and you can recharge the battery via USB when it needs it. You can even use the Pico while the battery is charging.

There are lots of suitable batteries available, from Pimoroni and others, but my current favourite is the Pimoroni Galleon; its rugged case reduces the risk of crushing or puncturing the LiPo.

So how do you hook up the LiPo shim to the Pico it's powering?

Pimoroni suggest two options.

Connecting the Pico

Both options involve some soldering. One is simple; the other, Pimoroni suggest, is suitable for advanced solderers. You can connect the Pico to other components via female header sockets or male header pins.

The simple option

You want to connect two layers of PCB, one for the Pico and one for the Shim. You can use stacking female headers like this:


Image courtesy of Pimoroni.

You get a bonus (sockets on the top of the Pico, giving you extra connection options), but there are two disadvantages: your project is now quite a bit bigger, and you may find it fiddly to press the bootsel button if you need to.

There's an easy fix to the bootsel issue; you can use mpremote's bootsel command, or run machine.bootsel() from a REPL.

If space is a problem, you have another option.

For advanced solderers

From the Pimoroni site:

"Alternatively, if you're ambitious in the ways of experimental soldering, you can try soldering the Pico and the SHIM both to the short end of your header, back to back. This method makes for a much more slimline Pico/SHIM package which works nicely with Pico Display, but you'll need to make sure your solder joints make good contact with the pads of both boards and the header."

Here's how you do it, in words and pictures.

Step 1

Push 2 rows of 20-pin male headers part way into a breadboard and place the shim on top. Make sure that the

button of the shim is located below the shim and below where the USB connector of the Pico will be.



Step 2

Lay the Pico/PicoW on top of the shim


Step 3

Solder the corners of the Pico to the headers and check that everything is correctly aligned.

If not, it's fairly easy to melt the solder on the relevant corner and fix the alignment.


Step 4

You may want to check the underside of the Pico as well as the top.

Step 5

Now solder more of the header pins, filling the castellations at the side of the Pico and shim with solder.


Step 6

If all is well you should see a white LED, showing that the Pico is powered, and a red LED, showing that the battery is charging.

NB: You may need to push the button on the end of the shim to turn the power on.

Well done! You are now an official advanced solderer :)

Checking the battery voltage

If you're using a Pico you can easily use an OLED display to show the state of the battery. There's a useful program to do that on the Pimoroni website, but it won't work for the PicoW without a change. That's because it uses ADC29 (also known as ADC3) to monitor the Vsys voltage.

You can do that on the Pico W, but you need to pull P25 high which disables wireless. Since I wanted to use MQTT over a wireless connection to monitor the battery, I had to find an alternative solution.

A couple of the projects I had in mind used two of the three normal ADC pins on the pico, but none used all three., If your project doesn't need three Analogue inputs, you can use one to monitor the battery voltage.

That needs a little care, as the GPIO pins on the Pico (including the analogue pins) should not be connected to more than 3.3 volts. A fully charged Lipo might be outputting 4.2 volts, which would damage the analogue pin.

Fortunately there's an easy solution.

Using a voltage divider.

You can use a voltage divider to reduce Vsys to a safe voltage. I used two 10K ohm resistors, but 100K would be better, as that would reduce battery drain.

Then you can safely read the adc output and convert it to a voltage. Remember to multiply it two to compensate for the divider!

Here's the breadboard layout for the divider:


and here's the schematic:


The code

# This example shows how to read the voltage from a LiPo battery
# connected to a Raspberry Pi Pico via the Pimoroni LiPo shim for Pico

# should contain your network id and password in this format:

SSID = 'your Wi-Fi network id'
PASSWORD = 'your Wi-Fi password'
MQTT_HOST='broker url'

from machine import ADC, Pin
import time
import random
import network_connection
from umqtt.simple import MQTTClient
from secrets import SSID, PASSWORD, MQTT_HOST

# connect to wifi
# this may take several seconds

network_connection.connect(SSID, PASSWORD)

CLIENT_ID = 'pico-lipo-monitor-%d' % (1000 + random.randrange(999))
mc = MQTTClient(CLIENT_ID, MQTT_HOST, keepalive=3600)

adc2 = ADC(28)
conversion_factor =  2 * 3.3 / 65535

while True:
    voltage = adc2.read_u16() * conversion_factor
    message = 'Lipo voltage: %f' % voltage
    mc.publish('lipo', message)


That's how you can connect the Pimoroni Lipo shim for Pico and the Pico W, and how you can then monitor the LiPo battery voltage.

I'll be publishing the hardware and software details of the weather station project in my forthcoming guide to the Pico W. Read more details here.