A little Raspberry Pi history

a pre-alpha version of the Pi (reproduced with permission)

A few days ago someone pointed out that it was eight years since Eben Upton's first blog post on the Raspberry Pi website.

The site had actually been live for a while (from 6 May 2011), with just enough information to get me and others excited about the potential of the Pi.

The website explained that the goal was to produce a low-cost Linux-capabable ARM-based board for educational purposes.

That was interesting, but not world-shaking. There were already a few hobbyist SBCs (Single Board Computers), but they weren't cheap. I think the BeagleBoard was over a hundred pounds, and  you needed some specialised hardware to level-shift the 1.8V GPIO pins to a useful voltage.

So how much was the Pi going to cost?

According to the website The expected price is $25 for a fully-configured system.

Back then, that was almost unbelievable. Indeed, to many, it was unbelievable.

The early days of the Pi were dogged by negative comments reflecting two schools of thought.

One said the Raspberry Pi foundation would never deliver a working system at anything like that price; the other said that no-one would buy it even if they did.

Sales of over 15 million have proved both fears wrong :)

When mu is not enough...

I'm working on a micro:bit project book at the moment, and the current chapter covers a Quiz runner application.

It needs multiple micro:bits. For testing I need at least three, one for the Quiz runner and one for each of two competing teams.

Although the micro:bits have different roles, they are all running the same software. The Quiz runner identifies her micro:bit by pressing button A; her micro:bit then sends a radio invitation for the other micro:bits to check in, team by team.

From then on each micro:bit knows its role and can behave appropriately.

 

Mu and PyCharm

 

I normally use Nicholas Tollervey's excellent mu editor for MicroPython development, but the Quiz runner program is more complex than most.

I am using Git to commit versions as the development progresses. Mu's minimalist design makes it very easy to learn. To keep it simple it has no integration with version control applications like Git, so I am using JetBrains' PyCharm.

A few weeks ago I started using a micro:bit plugin for PyCharm which allows me to develop and deploy to a micro:bit. That means I can use PyCharm's integration with Git and GitHub to keep track of the code as I develop it.

 

uflash and the micro:bit

 

The Pi farm programming multiple micro:bits

PyCharm with the plug-in lets me deploy code to a single micro:bit but this particular application requires me to deploy the same code to three or more micro:bits whenever the code changes.

I'm using Nicholas' uflash for that. uflash ( pronounced microflash) can transfer files from a laptop, desktop or Raspberry Pi to an attached micro:bit. I have written a very short shell script which I run on my desktop development environment.

It uses scp and ssh to copy a Python file to two or more Raspberry Pis and then run a uflash command to install that file on each micro:bit. If you need to install software to multiple micro:bits (perhaps for a workshop or class) it's a real time-saver.

If you want to follow progress on the micro:bit project book, subscribe to my free Physical Computing newsletter. You can sign up here.

reggie - readable regular expressions in Python

Regular Expressions are powerful but they can be hard to tame.

There’s an old programmer’s joke:

A programmer has a problem, and she decides to use regular expressions.

Now she has two problems.

Unless you use regular expressions regularly (sorry for the awful pun!), they can be

• Hard to read
• Hard to write
• Hard to debug

reggie makes regular expressions readable and easy to use.

I've used regular expressions for a number of projects over the years, but the idea for reggie came about a decade ago.

I was working on a project where we had to parse CDRs (Call Detail Records). These are text files that Telecommunication Service Providers use to identify calls and other services billable to their customers.

The CDRs we received were CSV files with a complex format and we decided to use regexes (Regular Expressions) to verify and interpret them.

We liked regexes, but they were less popular with our business analysts. The BAs were happy with our Java code, but they found the regexes much less readable.

Nat Pryce and I came up with a simple DSL (Domain Specific Language) which we used to describe and then analyse the CDRs. That made it much easier for our BAs and Testers to understand our code, and developers joining the team mastered the code quickly.

The Java syntax was a bit messy, though. These days I do most of my development in Python or APL, and a couple of years ago I realised that Python's syntax would allow a much cleaner implementation, which I called reggie.

It's been useful for several projects, and I decided to share it. You can now use reggie for your own work; it's available on GitHub.

A simple example

Here's the solution to a classic problem, converting variants of North American telephone numbers to international format.

(The code is in the examples directory).

from reggie.core import *

d3 = digit + digit + digit
d4 = d3 + digit
international = name(optional(escape('+1')),'i')
area = optional(osp + lp + name(d3,'area') + rp)
local = osp + name(d3,'exchange') + dash + name(d4,'number')
number = international + area + local


def convert(text, area_default='123'):
    matched = match(number, text)
    if matched is None:
        return None
    default(matched, 'i','+1')
    default(matched, 'area', area_default)
    return '{i} {area} {exchange} {number}'.format(**matched)

print(convert('(123) 345-2192'))
print(convert('345-2192'))
print(convert('+1 (123) 345-2192'))

Here's the output:

+1 123 345 2192
+1 123 345 2192
+1 123 345 2192

A CDR Example

The next example is inspired by the original Java-based project.

Here's a simplified example of the format of CDRs (Call Data Records) for UK Land Lines and a raw regex (ugh) that could be used to parse it.

Type, Caller,        Called,        Date,       Time,       Duration, Call Class
N,    +448000077938, +441603761827, 09/08/2015, 07:00:12,   2,
N,    +448450347920, +441383626902, 27/08/2015, 23:53:15,   146,
V,    +441633614985, +441633435179, 27/08/2015, 18:36:14,   50,
V,    +441733360100, +441733925173, 12/08/2015, 02:49:24,   78,
V,    +442074958968, ,              05/08/2015, 08:01:11,   9,         CALLRETURN
D,    +441392517158, +447917840223, 22/08/2015, 10:14:39,   2,
V,    +441914801773, ,              18/08/2015, 17:29:50,   7,      ALARM CALL

Regex: (one long line, split for readability, not a pretty sight!)

(?P(N|V|D)),(?P\+[0-9]{12,15}),(?P(?:\+[0-9]{12,15})?)
(?P[0-9]{2,2}/[0-9]{2,2}/[0-9]{4,4}),(?P[0-9]{2,2}:[0-9]{2,2}:[0-9]{2,2}),
(?P[0-9]{1,1}),(?P[A-Z]{0,50})

For educational purposes, I've adapted the CDR specification/sample data from the official standard provided by FCS.

I've also omitted a lot of fields and inserted spaces to clarify the layout. The spaces are not present in a real file and the regex examples correctly assume there are no spaces in a real CDR.

Below it you can see the definition of the format using reggie.

from reggie.core import *

call_type = name(one_of('N','V','D'),'call_type')
number = plus + multiple(digit, 12, 15)
dd = multiple(digit, 2, 2)
year = multiple(digit, 4 ,4)
date = name(dd + slash + dd + slash + year,'date')
time = name(dd + colon + dd + colon + dd,'time')
duration = name(multiple(digit),'duration')
cc = name(multiple(capital, 0, 50),'call_class')
cdr = csv(call_type, name(number,'caller'), name(optional(number),'called'),date, time, duration, cc)

Now you can try to parse some CDRs. The last one is wrongly formatted.

Parsing a well-formed record returns a Python dictionary.

Parsing an ill-formed record returns the value None.

So running

print(cdr.matches('N,+448000077938,+441603761827,09/08/2015,07:00:12,2,'))
print(cdr.matches('V,+442074958968,,05/08/2015,08:01:11,9,CALLRETURN'))
print(cdr.matches('Rubbish!'))

will print

{'caller': '+448000077938', 'call_type': 'N', 'date': '09/08/2015', 'called': '+441603761827', 'duration': '2', 'time': '07:00:12'}
{'caller': '+442074958968', 'call_type': 'V', 'date': '05/08/2015', 'duration': '9', 'call_class': 'CALLRETURN', 'time': '08:01:11'}
None

Other Applications

I've used reggie to build parsers for simple languages. Regular Expressions have some limitations - in particular, you can't use standard regular expressions to analyse recursive grammars. But there are a surprising number of applications where a DSL without recursion does the trick.

I've recently been working on an emulator for the venerable (and wonderful) PDP-8 computer, and I needed to create a Python assembler for PAL, the PDP-8's assembly language. The assembler uses reggie and it's about an A4 page of fairly simple code.

Breadboarder

I've also been working on Breadboarder, a DSL for documenting breadboard-based electronics projects. At present you have to define your project using Python code but I think I can use reggie to create a natural-language version.

Morten Kromberg and I have also been experimenting with an APL version of reggie. APL syntax is well suited to reggie, and the experiments look very promising.

We've included our current APL prototype in the main GitHub project and I'll blog about that once the code has stabilised.

In the meantime, take a look at the Python version and leave a comment to let me know what you think of it!