B2B Software Must Answer For Its Sins

Today I have been subjected to a software experience that was so intensely terrible that I sat down and wrote for hours after not writing for months.

Some Non-Technological Technical Background

A permit to work system is how large industrial sites control what's happening on their watch on any given day. A large chunk of maintenance centres around this.

You wouldn't want to paint, inspect, service, or otherwise be inside or nearby a big piece of heavy machinery while it's running, therefore, you must ensure it isn't running before you do any maintenance on it. You also need to make sure everyone you send down into the equipment also comes back out of the equipment, so you don't leave Oblivious Larry trapped inside the chemical tank when it's time to box it up and flood it with chemicals or on top of the roof after you dismantle the scaffolding. And you need a way for the work crew/ site person who hired the contractors to explicitly announce that "yep, the job is done, everything is back the way it's supposed to be," so no one leaves any open manholes for people to trip and fall into.

A permit to work system is any kind of management system that achieves these goals. It became a priority in the late 19th / early 20th century, as industrial sites got bigger, and workers were more frequently total strangers and also people got angrier about people dying in industrial accidents. At one point, industry was small and slow enough that you could just use your own brain and eyeballs to figure out what was going on and just Not Be In The Way; when everyone knows each other you can also just talk to everyone else and know what's going on generally. Also, internal combustion engines hadn't been invented yet, so there's little risk of you doing some work and then Suddenly Random EWP/ Crane/ Front Loader/ Cement Pourer Appears.

Enter the 21st Century, when the maintenance workforce is mostly contractors, basically all strangers, and the site staff is like a dozen people who have all independently hired a bunch of contractors to do a bunch of jobs (at the better sites, the dozen people do talk to one another and proactively manage any clashes). Anarchic self-management does not work here because of all the people involved; the PTW system demands everyone submit a form stating who they are, who hired them, what they're doing, when and where, and are they going to be bringing in special equipment or doing anything particularly tricky that everyone else should be aware of?

Could This Be An App?

The permit to work systems that have been used since the 60s (or maybe even earlier) were typically implemented on a series of paper forms. Typically, the forms would all get submitted to a single office on site, which may have one specific person who has the Final Say on whether work goes ahead.

The person usually has to look over the Overall Plan and de-conflict everything. No, we can't be breaking a flange and venting methane here while someone else is doing oxy-cutting, we'll get an explosion. If someone is shooting x-rays over here to look inside the pipes, then no one can be in this area while they're doing that. If someone is climbing that structure to do a paint job, then that other crew can't be wiring up instruments underneath them. This is typically a super experienced ops guy, and this is a hard job, with high stakes.

Also, this office checks the work party daily sign in/sign offs. Al, Bob, Charlie, and Dom sign in at 7AM to clean a tank. If they do not return to the permit office by 6PM to sign off, the permit officer will ring the supervisor on the permit, and if no one answers then they'll check the paperwork which tells them where those guys are supposed to be, and they'll go down to the tank and see if they're still there and why they didn't come back and are incommunicado, like maybe the tank has filled with toxic gases and everyone's knocked out or something¹. If there's one specific person who hasn't signed off, they'll ring the supervisor, and the supervisor will either say "that guy had to leave for a family emergency, sorry bout that, I can prove that he's off site now", or, "oh shit we left Bob in the tank."

Since most workplaces use computers to do all the admin now, you'd think it's about time we started making apps for these. Most of the documents in permit packs are now compiled on computers; it would be good to skip the step where you repeatedly print out and scan these things. There's a lot of these tasks that would make sense to automate - checking that everyone actually signed off is an excellent use for technology, and it would be convenient for the worker if it's accessible on your phone (just hit the "yep I'm safely off site" button!), as well as an automated alert to supervisors prompting them to check if they've Left a Guy in the Pit. Computers are great at pulling up Every Record matching some keyword; it would likely beat a human manually reading all the paperwork and check which ones are located where. You could put GIS data on a computer-based permit system; you could upload photos without printing them out. You'd never accidentally drop one down the back of a desk (or into the sea). Lots of genuine advantages. 

I had the distinct privilege to be part of the contractor-side roll-out for one of these today. Unfortunately, so far, it has failed to deliver on most of these very admirable goals.

Signing Up To a Totally Normal Portal

It started with the sign-up flow. It looked normal. I entered my email, let my password manager generate some gibberish, and hit the register button.

Error: User does not exist

I checked my email, confirming that I did in fact get the system-generated email stating that I've been added to the platform by the admin and that I was in fact supposed to be signing up. I check the spelling of my email. I check whether its got the .au suffix. I try again.

Error: User does not exist

Well, that's probably not an error on my end. I give up and figure it'll be sorted the next day, when I'd be attending the onboarding session² with the client, trusting them to sort me out.

I roll up to the onboarding. I reproduce the error. The client rep, a lovely guy in their occupational safety team, swears, and mentions that he'd been dealing with these errors for four weeks. He dicks around with something on his side, which looks like some sort of database, and we check that everything is spelled correctly. It is. We try a few random things, and I report that mysteriously, I can log in now.

We high-five and celebrate... and he goes to troubleshoot the three other guys who are in the onboarding session with me (those guys were landscapers, by the way).

I have my laptop open, so I read the next sets of instructions, which tell me to download an authenticator app, which I do, and a second app for accessing the permits. Apparently the browser-based thing I signed up for is the 'contractor portal'. I'm meant to do job planning and create the permit from a desktop browser, wait for someone on the client's end to approve it, then roll up to site with the app on my phone - the app doesn't create permits.

Fair enough! Planning is in fact mostly a job to be done before you roll up to site. The client OHS rep then says that currently, the contractor portal doesn't support uploading documents yet.

That is... not great. Permits are designed to be a concise, standardised form that's easy to glance at; they're not supposed to have exhaustive detail on what's been planned and what the risks are (that would be the scope and the risk assessment), but said detail should still be accessible.

"There's a workaround," the OHS guy adds. Apparently the client-side version of this thing can upload documents, so the specific person hiring us is meant to send us a link to upload our documents.

That's a little bit janky, but if it works, it works, and it sounds like it's on the roadmap. My hopes for this app have been dampened slightly by the 'user does not exist' thing but they have not yet been utterly dashed just yet.

OHS guy creates our roles, and the other three people continue to struggle through registration. Unfortunately, we do not succeed in reproducing the mysterious conditions that let me log in; OHS guy ends up manually entering passwords on the client-side portal thing on each of their profiles, and asks them reset their passwords later. This all takes a while. We start chatting about where to get good pizza around town.

I mean, this isn't our first janky work IT situation, so the mood is still relatively light. The landscapers are friendly, and the OHS guy is nice.

Logging Into A Totally Normal App

While we chat / log in, my second permit access app finishes downloading, so I open it.

I get hit with a loading screen. It lingers for a while, then an error message pops up.

There is a valid internet connection but we are unable to communicate with the API.

Um, okay. I start getting nervous. I hit retry. A few seconds later, another error box appears.

An error has occurred. Please go to the Settings page and ensure that the details are correct.

The three other guys successfully log in. The OHS guy mentions that today actually went better than some of the other sessions, because it only took about forty minutes to get everyone logged in.

"When you open the app, it'll freak out for while, this is normal."

Hmm. I check my calendar and realise that the onboarding session is two hours, and that we've already spent nearly an hour of it struggling with the logging in.

"So what you wanna do is wait for it to load and then type this address into the server field-"

Could they not have made it a drop-down list?? We're just going to get this app to connect to arbitrary servers? I guess?

"Then turn on enable self-signed certificates."

I don't have much interest in computer stuff, but this made me a little bit nervous. The app itself generated a pop-up about it being less secure. I did a bit of Googling and a bit of asking friends after work.

Cybersecurity Interlude: I Learn Why Certificates Are Important

I learned that self-signed certs are very insecure. For the software-inclined reader, this is probably like teaching you to suck eggs, but nevertheless:

The internet can be used to make computers (or phones or devices) talk to each other. In most applications, it is vitally important that the computers know whose computer(s) they're talking to, because there's quite a lot of stuff you'd tell your bank or your doctor that you wouldn't necessarily be telling some nefarious-looking rando on the street.

Typically, there's something called TLS which does this - each computer gets a secret code with lots of digits, which is used to encrypt the data they send each other. The other computer must have the matching secret code to decrypt the data (it's more complicated than that of course, but this is the simple version as I understand it).

But wait, where do these computers get their secret codes from?

Well, they can generate their own secret codes. This is basically just letting the other computer roll up to my phone and say "here's the secret code to send stuff to me." This will let my phone securely send things to that other computer. That's great!

.... Hang on, wait, is that other computer actually who they say they are, though? Who have I been securely sending my data to?

This is a Classic Hack which does not require breaking any encryption; all the hackers need to do is to claim to be the computer your phone is looking for, and tell it how to securely talk to said hacker, and because your phone is dumb it will merrily tell the hacker everything.

This problem, of course, has had a standard solution for a long time. A trusted third party called a Certificate Authority hands out the secret codes. The CA goes "you're who you say you are, they're who they say they are, you can both have the secret code"³. Traditionally, this service used to cost money, like twenty whole dollars for several years or something, but LetsEncrypt has been providing this as a totally free service since.... 2016.

According to my more knowledgeable friends, there's basically no reason to use self-signed certificates in 2026. A lot of people have worked on this project and championed it, so that developers aren't priced out of practices that will protect users.

Aaaaand these guys aren't using those tools, I guess. Even though it costs them nothing.

They're Asking Us To Install Malware??

It's probably not literally malware⁴; the app doesn't ask for a lot of permissions, so there's no way for it to directly take over the entire phone. However, typical use will involve names and phone numbers of users, and possibly attachments. It's likely quite doable to intercept the traffic, make a perfect copy of the permit server, and replace every single PDF with a dodgy version that will also install ransomware as soon as someone opens it on their phone.

I have been advised to install it on a dedicated work phone if I need to use this app.

Just for fun though, I turned off self-signed cert (or as I now call it, Malware Mode) and tried logging in again. It worked fine. I have even more questions now.

I am completely mystified about why the client is requesting all of their contractors and most of their ops staff to basically leave a backdoor into their phones. That poor OHS guy was almost certainly given these instructions explicitly by the software vendor, and lacks the technical background to understand what he is asking everyone to do, which is fair because his main job is to make sure people don't get trapped inside tanks at work.

I am lucky. I have enough tech knowledge for the phrase 'self-signed certificate' to set off alarm bells in my head because I hang out with nerds in my spare time. The majority of the people being told to download this are tradies, who possibly haven't finished high school. They're unlikely to be own work phones. They're unlikely to be able to argue their bosses into giving them work phones. I pay a bit extra to have a dedicated phone number just for work; many of them don't.

The hypothetical victim is likely to be a 60 year old electrician who goes by Toastie will lose his life savings to a scammer because his full name and phone number got leaked onto the darkweb by badly designed apps like this.

Y'know, the exact sort of thing that the good folks at LetsEncrypt and their supporters are trying to prevent with this work.

More Things Break

Back to the onboarding session. I do as I'm told, and a pretty normal looking login screen eventually appears. For some ungodly reason, the username field wants First Name Last Name this time, not email address like for the previous one. We discover that while they were downloading the apps, the database had inexplicably flagged two of the other guys as 'discontinued' and was automatically appending the date to their surnames, breaking the second login flow.

The OHS guy fixes this. He mentions offhand that he has been running these onboarding sessions for the past four weeks, and will keep doing them for the rest of the year. I wince. This man has been absorbing the full brunt of this terrible app for the past month, and he does not deserve this. No one does, but he's one of the better humans working for this organisation.

Everyone manages to login to the app, finally. The OHS guy runs through some fairly straightforward instructions on some PowerPoint slides, as well as some reminders and clarifications of the OHS policies, and then we start on the 'practice' phase of the session, where he creates test permits for us to practice using.

I discover that I cannot log in to the test environment (although my production login appears to work). OHS guy decides that I'm probably tech savvy enough to figure it out myself when it comes time to the real thing, and I look over the shoulders of the landscapers (this is hard because phones are small and landscapers tend to be large people).

When one of the landscapers signs on as permit holder / supervisor, his app freezes. It does not unfreeze after rebooting the app twice. He resigns to wait it out and watch over the shoulder of another landscaper. The OHS guy takes the chance to demonstrate another workaround: logging into another device.

In between waiting for things to load / unfreeze / etc, I mention that I heard the launch was supposed to be last month, but perhaps if it's only a month late, maybe they could have done more internal testing. The OHS guy reveals that they started work to digitise the paper system four years ago.

Holy shit - Four years?!

The earlier mood has now soured. The landscapers are distinctly frustrated, although everyone is still very nice to the OHS guy, because we all know it really isn't his fault.

This Is Why We Hate Programmers

We are now more than ninety minutes into this exercise; paper permits typically take around five to ten minutes to issue if there aren't specific problems with the scope, and most of that time is usually spent queueing behind the twenty dudes signing onto their hydrodemolition permit.

I have been issued moderately complicated permits in fifteen minutes, and the majority of that time is spent discussing the actual job. Approximately zero of that time is spent wrestling with paper, which lives in a specially printed book that makes two carbon copies every time one is issued. I mean, occasionally I'd had to wait for someone to scan or print documents⁵, but I have never, ever, had the process take this long.

Four whole years of god knows how many software people doing Software Things, delivering... whatever the fresh hell this is, and in all likelihood, each being paid more than any of the people sitting in this room - an inspector, an OH&S adviser, and three landscapers. The people who would be actually using this fucking thing. Concreters, painters, divers, electricians, plumbers....

The thing is, if any of us delivered something that was as substandard as this app, we'd be fired. The landscapers especially. It would be hard to get paid if the grass is dead. While there is bad performance in my own field as well (I recently opened a report where the job was tendered in 2022, the site was inspected in 2023, and the report was submitted in 2025, good lord) - generally poor performance is acknowledged to be poor performance. There is a reason why my boss won the job from the "takes an entire year to write a report" consultancy.

No other sector consistently gets a pass for always being late and delivering broken things, and this is still true for things that are done by large orgs for a captive userbase that can't complain too much, and transport infrastructure. Sure, they can go over budget and be a bit late, but the end result has to actually work!

We resent the software engineers who weren't in the room, who will never be in these rooms, or out on site, doing the work, using these products. We resent that they are not held to the same standards as we are. We resent that our performance - our ability to do our jobs and deliver our services on time and on budget - will be materially impacted by this piece of shit app, but we will not get this kind of grace, because for some goddamn reason, the organisation is lavishing it all on the software people.

Why are they allowed to suck this hard at their jobs???

I Do Not Care For Software....

I don't really give a shit about the minutiae of software, despite somehow always ending up on the periphery of these types. I do not know what docker is. I don't have opinions about GUIs. Why do you care about linters? I don't care to find out. When there's a serious spirited debate about some way or another to implement some sort of programming thing, I have a strong urge to roll my eyes and call everyone involved a nerd. 

But introspecting on this, my boredom with tech stuff is different in tenor to my simple disinterest in, say, coin collecting. My tech boredom is rooted in fury. I'm surrounded by computers and software, all day, every day; but I am often inconvenienced by it. I do much of my learning from Situations, like "today, work got me to download a really shitty app and I need to find out if it'll give me a virus or leak my bank details".

I was already grumpy about how badly the app works. After I spoke to a few software people and found out how bad the security practice was and how preventable it was, I was even angrier. Learning about software is mostly an exercise in becoming aware of just how precarious and broken my day-to-day IT experiences are. It's upsetting⁶.

And then there's the "use at own risk" thing. As consumers, we have been thoroughly educated - don't re-use passwords, only send things to secured sites, etc etc. In fact, the app even had a pop-up when I enabled self-signed certificates which said something like "it's important to understand that this mode is less secure, please exercise caution."

The majority of the users of this app will not be able to decipher this warning message. I had to ask my friends for help - but most people do not have casual access to a bunch of cybersecurity specialists! But these sorts of pop-ups are now ubiquitous.

It's fair for my browser to throw up one of these if I clicked on a sketchy 'claim ONE MILLION DOLLARS NOW' ad, but when we start seeing these pop-ups on even normal legit apps, especially on software we have to use for work, I really don't think the people involved should be able to palm off their responsibilities with a shrug and a "we warned you that it's less secure".

I Really Wish I Did Care For Software Though

Opposite to how I feel about software is aviation. A commercial flight is a marvel of socio-technical engineering. We, as a species, are capable of building, operating, and maintaining the means of actual literal flight.

I like it because what I learn tends to fill me with confidence. I enjoy learning about the complex, interlocking systems that keep me safe when I'm up in the sky. I even enjoy reading through the crash investigation reports.

I don't feel the need to inspect the welds and rivets on airplanes before I get on a plane. I get to trust that aircraft is maintained by teams of people who know how important their jobs are, and regulated by people who are competent and motivated. Everything I have learned about aircraft, I have learned of my own volition, for fun.

I wish I could feel a similar level of enthusiasm about software, but I'm learning about TLS and CAs because I was nervous about downloading this crappy app for work. I learned about testing in prod and CI/CD when nationally, the phone network just stopped working (and that other time when an untested patch for an anti-virus software shut down a quarter of all computers globally).

Trust is a vital part of what makes societies actually work - because the world is already so complicated, and getting complicated still, that no one person can cover all of it. I'll inspect infrastructure; someone else will cure cancer.

It's a massive problem when we can't trust the software that we use every day. Individually excellent pieces of software exist - in the things that we can voluntarily install, like the music player on my phone. Most of the software that we interact with day-to-day, however, are B2B software. You, the user, don't get a say in what apps or websites or services the organisation uses when you interact with them as a worker, a customer, a citizen or a jobseeker.

I know people in tech, and as far as I can tell, they're just people, like the rest of us. There are people who are competent, and people who aren't; people who care a lot and people who are just doing bare minimum. Except this is how people are in every industry. The teenager assembling your burger at the Maccas may produce a somewhat squished and ugly burger but they very rarely hand over food that is actively hazardous to health, and the occasions where they straight up lose the order are also fairly uncommon⁷. We don't expect to get double-charged and missing fries every time we visit Maccas; we somehow do expect that every B2B software project will deliver some kind of trash fire.

I also don't think it needs to be like this. I think it starts with not accepting this state of affairs, and demanding better from the software that we buy and use - but that should probably be another post. This one's more than long enough.

Until next time!