The Evolution of Logging

In a previous article on Field Day logging, I focused on a networked logging solution using N3FJP ARRL Field Day Contest Log software.  A link to the article was posted on Facebook where a discussion ensued that included mention of other great logging packages. The discussion inspired me to go a little deeper into network logging starting with running the logging packages mentioned on a few virtual machines in my home lab.

This article is targeted for developers, those responsible for the technology behind club logging especially for events like Field Day, and those who are always interested in what is behind the technology curtain.

The Network Logging Landscape

N3FJP follows a client-server model where one of the PC's can do double duty as a server either using Microsoft File Sharing to share its local MDB file or running a TCP server process.

N1MM Logger+ uses a UDP broadcast approach by all participating PCs running N1MM Logger+. The broadcast can be unicast to the local subnet and/or directed to a list of IP addresses on local or reachable networks.

The TR4W logging software (which is Open Source) approach is similar to the N3FJP TCP approach but runs the server as a small separate application.

All applications mentioned make local scheduled backups of the collective logs and document methods of restoring logs in the event of outages. Each of these packages have a strong standalone heritage with networking "bolted on" over time. But in today's world network applications are no longer monolithic instead built as a collection of loosely coupled services to ensure high availability and data integrity.

Service-Oriented Architecture (SOA)

From Wikipedia

"Service-oriented architecture (SOA) is an architectural style that supports service orientation . . . well applied in the field of software design where services are provided to the other components by application components, through a communication protocol over a network. A service is a discrete unit of functionality that can be accessed remotely and acted upon and updated independently. SOA is also intended to be independent of vendors, products and technologies."

After spending some time looking at how networking is done with N3FJP, N1MM, TR4W, and Log4OM I think for networking they all need to adopt a loosely-coupled architecture. This architecture would have the logging applications continue providing standalone capability but any shared and peered services is via a message service bus. Peers may be the same or other logging applications sharing logs for dupe checking log protection. Shared services may be a common database server, chat, or some online log services (discussed further on.) 

Message Queue (aka bus service) enables applications to communicate with each other as peers exchanging information using a standard protocol. The applications themselves can be in different languages implemented on different platforms. Both N1MM and Log4OM do something close to this already but there is no guaranteed delivery given the use of UDP.

In order to provide message queuing and guaranteed delivery, a message broker would need to be part of the queue. Message brokers can validate, store, route, and deliver messages to the appropriate destinations with the message queue processing messages in the exact order in which they were sent and remain in the queue until receipt is confirmed. Separate queues can be created and maintained for service isolation such as logging, chat, and other service queues that trigger uploading logs to online log services.

Online Log Services

The most well known online log service is Log Book of the World (LoTW) run by the ARRL and free to all radio amateurs.

"LoTW’s defining role is to provide confirmation of two-way amateur communications with impeccable reliability and security. LoTW maintains a record of logged and confirmed contacts, and identifies confirmations associated with supported awards. . . While ARRL awards are a priority for support by LoTW, awards offered by other organizations may be supported by Logbook, subject to prioritization by the ARRL Administration and Finance Committee."

The standard formats for importing logs to this and similar service are ADIF and Cabrillo. All the logging applications discussed in this article have the capability to import/export these formats as well as upload to LoTW.

Another popular log service is online scoreboards. Logging applications periodically upload log information in XML format via a HTTP POST method to a website such as Score Distributor,  CQcontest.net, and Contest Online Scoreboard.

Applications and Platforms

Developers like to stay within the safety of their preferred integrated development and the commonly used libraries within it.  For example you may see support for MDB, SQL, SQLlite, databases but not NOSQL because the Microsoft ODBC driver used by Microsoft application developers does not support it. Exception to this are those who are developing for Azure Cosmos DB. Same said for networking. 

Since it is easier for developers to build to breadth than depth you see expansion from one to  a suite of applications with intent of becoming the go-to platform for your Amateur Radio needs.  Ham Radio Deluxe is one of the few Amateur Radio software platform successes out there to date evolving from free applications to a subscription model. Being the platform rather than just an application is similar to commercial enterprise software strategy. Be the platform for all other applications to build on to grow customer base and revenue.

The need to grow revenue through building customer base is the reason why I am pessimistic any Amateur Radio Logging software application that is not Open Source will embrace an SOA model and instead limit such interoperability to named applications. I get it, trying to make a living developing software for the Amateur Radio is far from lucrative and requires more passion for Amateur Radio than expecting to make a livable wage from it. 

Futures

Many licensed amateurs in recent years come from other technology communities where collecting achievements is their motivation. They get their license, don't do anything with it, and move on to some other challenge. Online log services could evolve to "gamification" of existing contests and perhaps introduce new corporate sponsored ones attracting these quiet amateurs who always up for a challenge.

I hesitate to say as much as tracking clicks and eyeballs are monetized on the Internet, tracking Amateur Radio contact statistics could help keep interest in developing/sell ing products to the Amateur Radio market as well further data to be referenced in spectrum protection more readily that just a handful of organizations.

73,

- Joe, NE2Z