Behind the scenes in building a SIGINT Linux distro

Introduction

Back in October 2021 I was interested in building a SIGINT platform. I logged my journey and published an article on HVDN at that time. Little did I know that journey would have led me to build and maintain a SIGINT build tool for Linux called SIGpi for the past two years with 28 releases during that time.

Inspiration

As the saying goes, "Necessity and is Mother of invention." It started with Steve K2GOG wanting to run the latest release of SDRangel on the Raspberry Pi. SDRangel is an awesome Open Source SDR and signal analyzer tool written by Edoaurd Griffiths F4EXB. If your a long time fan of the HVDN notebook, you already know Steve is a huge fan of SDRangel.

Often your Linux distribution of choice may not have the latest version of your favorite software whether available as a distro package, Snap, or Flatpak. In fact it could be many versions behind. Some authors will have package releases on their GitHub sites ready to download and install. If not and if its open source, you will be able to download the source could from their GitHub repo source and compile yourself. While authors may provide compiling instructions, you will find yourself down the rabbit hole addressing all the dependencies the software has first - libraries, drivers, specific versions, etc. The advantage in using packages that came with distros is the package manager will take care of auto-installing dependencies and their correct versions.

Fortunately SDRangel is well-documented providing detailed compile instructions. Once I mastered that I canned the instructions into a Bash script. and shared that with Steve. After getting rid of some bugs with my script we achieved success

Crawl

Compiling code and writing shell scripts was something I had done in the past but not to the degree of calling myself a developer. But working with SDRangel inspired me to expand on that Bash script and create a script that would install a series of drivers, libraries, and applications to create a SIGINT platform for my needs and anyone who had similar one - SIGpi 1.0 was born.

SIGpi 1.0 was really just one big bash script named SIGpi_installer.sh. SIGpi 1.0 installed drivers for RTL-SDR, HackRF, and PlutoSDR SDR devices and the Soapy SDR platform which provides a vendor-neutral API for SDR applications. With my compiling and scripting skills leveling up, I could confidently choose whatever latest SDR application I wanted for SIGpi and compile and package myself as long as it was open source and the source code available. But the cost of using the latest software versions is it could take up to four hours for SIGpi to install on a Raspberry Pi.  For Amateur Radio related applications I stayed with the releases available from the distro since little changes with them.

Walk

Less than thirty days later I released SIGpi 2.0. Besides support for LimeSDR and additional applications, it introduced a text-based menu system allowing you to pick and choose what applications and SDR drivers you want installed. The menu uses Whiptail - a dialog box tool for shell scripts which is included in most if not all Linux distros. It is used by raspi-config.

Adding applications requires ensuring their dependencies are installed as well. Since many applications have same dependencies, I broke SIGpi install into stages where the dependencies for all applications would be installed before installation of the applications themselves. The stages were:

• Run OS update
• Setup directories and desktop menu skeleton
  
• Install devices
• Install dependencies
• Install Libraries
• Install Applications
• Install Desktop Menu
• Reboot system
  

The staging model continues to this day but evolved from functions within a single install script to nested scripts called by the install script starting with SIGpi 3.0.

Run

To end users, the evolution of SIGpi from 3.0 to 4.1 varied in the applications supported. But much of the development was focused on the underlying architecture of the SIGpi build to facilitate ease of updates. First step was to simplify the development process in producing releases. The second step was to extend that so end users can perform application updates themselves through a simply command line process.

For SIGpi 5.0 I created a package management system akin to APT as used by Debian and Ubuntu. SIGpi management differs in that it is source neutral. Source could be a distro package, local package, Snap, FlatPak, or download/compile/install from source code.This started with maturing application specific nested scripts used by the SIGpi_installer into "SIGpi Package Scripts." The package scripts would be used by SIGpi_installer and a new command-line script called SIGpi - a front end to the package scripts.

Usage: sigpi [ACTION] [TARGET]
          ACTION  
                 install   install TARGET from current release
                 remove    remove installed TARGET
                 purge     remove installed TARGET and purge configs
                 update    check to see if new TARGET available
                 upgrade   upgrade TARGET to latest release

          TARGET
                 A SIGpi package

The update command checks the SIGpi repo if there is a new release. If the response is "update is available" then you can run the upgrade command and the target will be updated to the new release.

SIGpi update sdrangel

Update is available 

SIGpi upgrade sdrangel 

Fly

Compiling applications like SDRangel and GNUradio on the Raspberry Pi contributes to more than half of the install time for SIGpi. Time is the price you pay to run the latest versions. To improve the install time I realized I needed to uplevel my skills again with the ability to create Debian packages. After some research I discovered the checkinstall tool as a simple way for creating Debian packages from compiled source. For SIGpi 6.0 I created a number of Debian packages starting with GNUradio and SDRangel that could be download and installed as part of the SIGpi installation process. I had now significantly reducing the install time to just over an hour.

I updated SIGpi package management with hidden developer options to build Debian packages of supported applications.

Usage: sigpi [ACTION] [TARGET]

          ACTION  
                 build     compile and install TARGET from current release
                 package   compile and create Debian package only

          TARGET
                 A SIGpi package

SIGpi 6.0 also saw the deprecation of 32bit for 64 bit.

Idling

SIGpi started with support only on Raspbian OS (32bit) for the Raspberry Pi 4. When 32bit support was deprecated, SIGpi not only migrated to Raspberry Pi OS (64bit) but expanded to include Ubuntu on x86-64/AMD64 architecture. The change was not only in response to the RPi hardware "famine" that was out there, but RPI pricing was reaching the point for serious consideration of low-end Mini PCs. While we test for those platforms, know SIGpi should be able to run on most Debian-based AMD64 and ARM64 systems.

Throughout the journey I kept considering whether SIGpi should continue as the build kit it is or be a full distro where you download and install a full OS image like DragonOS or DigiPi. What I kept coming back to is releasing a distro image may be easier for me to support and develop but it would be at the expense of limiting end-user experimentation integrating SIGpi into their own creations.

Another consideration has been whether to start using Docker and create docker images for complex SDR applications like SDRangel. Again what may be easier for the developer would be at the expense of experimentation by end-users. Docker adds overhead that the end-user would need to be familiar with.

Land

Many open source projects come and go for a variety of reasons. The best reason for closing a project is when similar projects merge and leverage the best of each. But often a project owner loses interest or support and refers people to other open source projects.

It makes me happy to see all the stars and forks the SIGpi project has received to date. It keeps me thinking on what more I can do with the project that delivers the most value in ease of experimentation for SDR while minimizing efforts for end-users in building a platform that supports it. This is the goal as I begin development for SIGpi 7.0

Areas of improvement considered include:

• A better TUI menu system for installation and SIGpi package management.
• Documentation on the SIGpi package management system for end-users to add applications requiring compilation for using latest versions
• Continuous development/release rather than point release model
  
• Hardware Certification metrics for SIGpi
  

I hope this article serves as an inspiration for your own projects and helps with decision points in your own journey.

73,

Joe, NE2Z

 

Journey into Software Defined Radio Experimentation

A few years ago, I published an article asking (Is a Poor Man's 705 possible?

​At that time, Ashhar VU2ESE announced work on the sBitx - an Open Source Software Defined Radio (SDR) using a Raspberry Pi 4 (RPi4.) The sBitx would be the successor to his previous project, the uBitx, a software-controlled radio using an Arduino. A PDF was published providing in-depth technical detail of the sBitx and a GitHub repo shared of code developed to date. The uBitx and sBitx are great examples reflecting the evolution of transceiver kit building.

 

Software-Controlled Radios

Software-controlled radios use software only to control the radio. Early examples of this include the use of Programmable Interface Controllers (PIC) with the PIC16F84 being quite popular. The PICs were pre-programmed and undocumented making them "black boxes." Replacing PICs with end-user programmable microcontrollers (MCUs) like the Arduino and open source code made software experimentation more accessible to the kit builder.

The uBitx architecture reflects a well-documented entry level software-controlled radio. Software-controlled because for it to be considered a SDR, signal processing needs to occur in the digital/compute domain. The RF section of the uBitx is mostly analog using a double conversion superhet design. The piece that is not is the si5351 i2c programmable oscillator commonly available as a breakout board module.

Diagram 1 - uBitx Architecture Model

The si5351 and the rest of the RF section is controlled by an Arduino MCU. The Arduino also drives the display and connects to a rotary encoder for user input.

Building the kit is a combination of plugging cables and soldering connectors. You can run with the firmware included or download the Arduino source code, modify, compile, and upload your own firmware. The firmware is created using the Arduino language compiled with the Arduino Integrated Development Environment a cross-platform application installed on a PC or laptop. The Arduino has a USB port for uploading the compiled code from your PC or laptop. The Arduino language itself is based on C/C++. 

Subsequent uBitx versions updated the display from LCD to a touchscreen by Nextion. Nextion displays are considered Human Machine Interfaces (HMI) with their own programmable microcontroller embedded dedicated to the task. The Nextion display is programmed using the Nextion Editor. The editor is used for GUI design and uploading the code to the display.

As a software-controlled transceiver, it is important to understand the code that is controlling and the implications of any changes you make to that code since you could damage it. If you are new to the Arduino, it is best you become familiar with it, building simple projects first and getting familiar with the IDE and the libraries it uses. The same can be said with the Nextion.

 

SDR using the Raspberry Pi

For many the Raspberry Pi is considered a more relatable technology to work with its single-board computer (SBC) that runs full operating systems supporting simpler popular languages like Python. The promise of introducing a Raspberry into a SDR design is more options for experimentation.

Diagram 2 - sBitx Architecture Model

​Diagram 2 is a high-level architecture model of the sBitx. The sBitx still uses a mostly analog RF section, but the MCU is replaced with a single board computer (SBC) the RPi4.

In this design, audio is routed through to an audio codec board using a WM8731. In turn, the audio is routed to the RPi4 via the SPI interface using the i2s protocol. There is also a switch from the proprietary Nextion display to commonly available touchscreens connected via DSI interface of the RPi4. The Sbitx includes a Raspberry Pi OS image with all the applications required to operate it included. The applications are compiled C.

Running operating systems adds processing overhead and latency which limits how much signal processing you can offload to the RPi4. This is why an audio codec board is used and all the code that runs the RF and Display sections is compiled C.

The allure of using a Raspberry Pi is the familiarity many have with it as part of their radio stations for logging, digital modes, and SDR RX projects and installing those applications on the RPi4 used by the sBitx. But lower-level experimentation with signals and control remains in the domain of C level programming to address latency concerns.

Up to now we have been covering HF transceivers which require more discrete analog components for RF sections. What about VHF/UHF?

 

Building VHF/UHF SDR transceivers

Say what you want about the Baofeng UV-5R and its derivatives, but its design has contributed to experimentation starting with its core component, the RDA1846 single chip transceiver coupled to a microcontroller. While the microcontroller code is proprietary some people have either hacked similar radios to upload their own firmware (such as OpenGd77) or breadboarded began using shielded RF modules like the SA818 "walkie Talkie" module This module includes an embedded microcontroller programmed through a serial connection using AT commands reminiscent of the analogue modem days. The best representation of this in kit build is the Hackerbox #0096 Two meter kit with its high-level architecture modeled below.

Diagram 3 - VHF/UHF Architecture Model

The design includes a SA818 module connected to a ESP32-S3 microcontroller with display and pushbutton connections to it controlling channels, PTT, etc. The kit refers you to GitHub repos for source code and compiles it using the Arduino IDE.  As with the evolution of uBitx to sBitx you could replace the MCU with an SBC and Audio board to expand from RF and interface control to include applications for SDR and digital modes with display and controls updated to a TFT. With an RF module, your experimentation on the build proper is all software.

 

”Full Stack” SDR

The “software” in open source SDR has been targeting the lowest common denominator in languages for developers for experimentation. This includes C, Bash, and Python. When it comes to interfaces, developers have a closer relationship to web development than coding in C for displays. You can see this in practice if you have used one of the many WebSDRs on the Internet.

Diagram 4 - "Full Stack" SDR Architecture Model

In the diagram above the application and interface sections reflect the web technologies required for a web browser to be the interface. I wrote an application and interface a few years ago (radioDASH) that will give you an idea what this looks like at the code level plus another project implemented with a LoRa transceiver on a microcontroller (HASviolet-ESP32.)

 

Why no HF RF modules?

In the simplest of explanations, longer wavelengths at HF frequencies require larger discrete components for filtering transmission spurs on unintended frequencies. Hence many of the QRP SDR kits out there have full RF boards with the local oscillators (that include the Si5351) on those boards controlled by a microcontroller. True SDRs like the FlexRadio use direct conversion for RF and proprietary FPGA chips for signal processing hence their expense.

 

Where is this all heading?

The lowest common denominator in the evolution from PICs to SDR is firmware developed in C given the low latency needs for signal processing. With the evolution from PICs to MCUs, entrants like the Arduino have made programming more accessible with its IDE and community support in code shared. But going from MCUs to SBCs only adds the ability of running peripheral applications like digital modes, logging, and radio control on the same platform. Compiled C is still necessary for any latency sensitive control.

Only when you have RF modules with embedded controllers and well documented APIs are software skill requirements relaxed. As mentioned previously, the SA818 module is configured using AT style commands via a serial port. This should sound familiar to those using dial-up or early cell modems.

When it comes to SDR experimentation consider your goals and the skillsets required before selecting the architecture model you want to pursue. This is why software-controlled radios remain the best entry level into the SDR space.

73,

Joe NE2Z