Give yourself the gift of quality control

If you spend any time at all in the tech chatter space, you have probably heard a lot of discontent about the quality of software these days. Just two examples:

I can’t do anything about the cultural, economic, and social environment that cultivates these issues. (So maybe I shouldn’t say anything at all? 🙂 )

I can say that, if you’re in a position to do something about it, you should treat yourself to quality control.

The case I’d like to briefly highlight is about our infrastructure rather than a software package, but I think this principle can be generalized.

Case study: bringing order to a data center

After a series of (related) service outages in the spring of 2020, shortly before the onset of the COVID-19 crisis, we cut back on some expansionary ambitions to get our house in order.

Here’s a sample, not even a comprehensive list, of the things we’ve fixed in the last couple of months:

  • updated every OS we run such that most of our systems will need only incremental upgrades for the next few years
  • transitioned to the Slurm scheduler for all of our clusters and compute nodes, which has already made it easier to track and troubleshoot batch jobs
  • modernized hardware across the board, including upgraded storage and network cards
  • retired unreliable nodes
  • implemented comprehensive monitoring and alerts
  • replaced our old LDAP server and map with a new one that will better suit our authentication needs across many current and future services
  • fixed the configuration of our Jupyterhub instances for efficiency

Notice: None of those are “let’s add a new server” or “let’s support 17 new software packages”. It’s all about improving the things we already supported.

There are a lot of institutional reasons our systems needed this work, primarily the shortage of staffing that affects a lot of small colleges. But from a pragmatic perspective, to me and to the student admins, these reasons don’t matter. What matters is that we were in a position to fix them.

By consciously choosing to do so, we think we’ve reduced future overhead and downtime risk substantially. Quantitatively, we’ve gone from a few dozen open issue tickets to 19 as of this writing. Six others are advancing rapidly.

How we did it and what’s next

I don’t have a dramatic reveal here. We just made the simple (if not always easy) decision to confront our issues and make quality a priority.

Time is an exhaustible, non-renewable resource. We decided to spend our time on making existing systems work much much better, rather than adding new features. This kind of focus can be boring, because of how strictly it blocks distractions, but the results speak for themselves.

After all that work, now we can pivot to the shiny new thing: installing, supporting, and using new software. We’ve been revving up support for virtual machines and containers for a long time. HPC continues to advance and discover new applications. The freedom to explore these domains will open up a lot of room for student and faculty research over time. It may also help as we prepare to move into our first full semester under COVID-19, which is likely to have (at minimum) a substantial remote component.

IDE vs. text editing app vs. editor in a shell

I’ve been doing software development of various kinds for a few years now, using a variety of tools. In my experience, these are the tradeoffs between an integrated development environment (IDE), a standalone text editing app, and a text editor in the terminal. (I haven’t listed any examples here that you need to pay for.)

IDE

Examples: Android Studio, Xcode

An integrated development environment is exactly what it says on the tin: a complete development environment for a project featuring a mix of tools that work together to help you build your application. An IDE’s features go far beyond reading and writing files, and might include…

  • managing your project in its version control system
  • scanning for possible mistakes in code
  • displaying a preview of your application next to the text file that describes it
  • serial port monitoring
  • debugging
  • automated building

IDE’s are powerful and can provide real boosts to productivity and insight. The tradeoff is lock-in: learning one IDE doesn’t mean you automatically know how to use another, and each editor has a learning curve.

In practice that’s usually fine. If you are exclusively, say, an Android app developer, you’ll be well-served to learn Android Studio. It just won’t teach you much about using Eclipse, let alone Xcode.

Text editor, standalone app

Examples: Atom, Notepad++

Standalone text-editing apps can be extremely handy. They’re not as heavy (in terms of system resources and user experience) as an IDE, but they’re great for all the common tasks we’d expect of an editor: find-and-replace, viewing the outline of a project in the filesystem, remote editing by SFTP, and more. Some editing apps, like Atom (which I use), support installing extensions that let you supercharge them to an almost arbitrary degree.

Good text editing apps can do all you need it to do and more. They may be the best choice for many users in many cases. That hasn’t been the case for me, though: I tend to experience them as an unhappy medium between horsepower and simplicity, and I don’t use them as often as the other two categories in this post.

Text editor in the shell

Examples: Vi, Nano

Vim is my fondest friend in the world of dev environments. I learned it within my first semester studying computing. While it has a learning curve, its keyboard shortcuts are simple and powerful. It is a widely-used example of an in-shell text editor – one that you run by typing its name in your shell followed by a space and the name of the file you intend to edit.

vim helloworld.txt     # 2 tokens, no waiting

This category tends to attract strong partisans. Stern computer science people on the Internet often emphasize vi for its near-guaranteed presence on Linux systems, for example. I like vim. Others like nano. Try them, and see what works for you.

There are some great advantages to using an editor in the shell:

  • Simplicity: An in-shell editor is sparse. You don’t get error checking, colorful little GUI buttons, simulators, design previews, or any other IDE feature, but that’s often unnecessary. (In most cases you can add font coloring, which is handy.) What you get instead is the ability to open and examine any file you can imagine – even if it’s a binary file and all you see is gibberish – in a predictable way.
  • Fast: From the POV of the user, an in-shell editor typically loads as fast as you can type. By comparison, IDE’s and editing apps have long load times – and sometimes long read and write times. This speed has the side effect of making shell editors operate similarly no matter your hardware, something that’s not true of either of the other two options.
  • Real big files: It can handle gigantic files better than GUI applications including IDE’s can.
  • Skills that scale: An editor in a shell makes you start learning how to use a shell. Knowing at least the basics of operating at that level is a solid upgrade to your development skillset. (The biggest confounding variable here is the type of shell you work with. I, like most people, just use the Bash shell. Experience could be quite different using ZSH or TCSH.)

Shell editors aren’t for every person working on every project, but I use mine several times daily because “it just works”.

(I’ll throw in a kind word for GNU Emacs here. It can be run as either a shell client or an editing app. I haven’t used it for years, but some people really love it.)

If the environment isn’t an IDE, building and/or running the edited software can be done by running your compiler or interpreter in the terminal. This isn’t as good an approach for something like an Android app, which needs to run on a device or at least a simulator, but it works for many applications.

So which do I pick?

I’m a fan of simple rules for choices like this. When it comes to development environments, try (in this order):

  1. The one everyone else working on the project with you uses. This is how I started working with both vim and Android Studio.
  2. The one you’re most comfortable with.
  3. If you’re early in your dev career and don’t yet have a comfort level with any of these, don’t be afraid to try several or all of them – it’s served me well to be able to shift between them from time to time.

Your mileage may vary on all of this, but these are the patterns in my experience.

Searching, sorting, and “Inbox Zero”

Two fundamental computer science problems that anyone who’s studied the topic will recognize are searching and sorting. These are intuitive enough at a high level:

  • Search algorithms scan a bunch of data to find what you’re looking for
  • Sorting algorithms take a bunch of data and re-arrange it into some order (that may be alphabetizing, grouping, labeling, whatever)

We use these principles outside the classroom as well. Consider email. The modern “knowledge worker” is inundated with emails, and there’s a certain subculture that constantly strives for “Inbox Zero” as a solution to the problem. I would argue that’s not the optimal strategy for most people, most of the time.

Inbox Zero is a strategy that’s heavy on sort and light on search. You want to classify emails, put some in folders, capture the information from as many as necessary, and delete as much as possible. That’s often useful, but a little bit of upfront work can handle a lot of that for you, and when you take those steps the Inbox Zero approach becomes much less attractive than it superficially appears.

As an example, here’s my strategy, which consists of a couple of sorting strategies and then (mostly) letting search handle the rest.

Sorting

The first sorting mechanism I have is the simplest one: unsubscribe to as many lists as you can. That’s great and for most commercial services I’ve done it, but I also receive emails from…

  • the Beowulf cluster mailing list
  • auto-pay services
  • the Earlham CS admin mailing list
  • the Earlham faculty mailing list
  • the ECCS GitLab instance if someone pushes an update
  • our wiki when someone makes a change

I don’t need every single message in all of those categories, but I know I’ll need or want some of them, so I can’t just delete them. That’s where filters come in.

I rely on (entirely too many) email filters on my Mac Mail client to handle preprocessing for me. Each filter applies a set of rules to incoming emails based on certain information in the email, often sender or subject contents. It takes many minutes/few hours to set up initially, but adding to it incrementally is quick and simple after. For that reason, it’s most of the sorting I ever do.

Those two tactics – cutting subscriptions and running filters – account for all of the sorting I do more than once every couple of months. That’s because I want the power of…

Searching

The major downside of deleting emails (especially if you then clear the trash, as I imagine most Inbox Zero people want to do) is that you can no longer search the content. Even if you think you’ve written everything down and downloaded every document, there’s a chance that something you consider unimportant now will become important later. Your ability to remember the context of an exchange depends on how well you’ve captured the details in your notes, and you may not know all the details you need for a long time.

The ability to search every message a human being has sent me in the last several months is handy. It’s not something I do all the time, but it has high value on the occasions when I do it. I’ve recovered many documents, resolved ambiguities during meetings, and revived forgotten but important conversations by doing this.

Why Inbox Zero?

Space is cheap, (human) time is expensive, search is extremely useful, sorting is only modestly useful… and yet we spend a lot of human time rearranging messages. Why?

I think the answer is obvious: It looks and feels nice to have a squeaky-clean inbox. An inbox full of messages feels like clutter, and clutter is a stressor.

That’s a perfectly valid reason! It’s why, a couple of times a year, I reach for Inbox Zero myself. I’ll apply a bunch of sorting to my inbox: deleting, archiving, moving things to new folders, etc. It’s just not necessary most of the time. My attention, like most people’s, is much better devoted to getting work done than to maintaining a clean inbox.

Most of the time, I’m happy to search 773 emails rather than sort 773 emails. Go to Inbox Zero because you want to, not because you think you must.

UPDATE: I promise I wrote and scheduled this post before I saw the most recent (Friday July 26) XKCD cartoon but I’m happy with the serendipity:

Defining your role in a workplace

There are three aspects of being a member of a group, team, or organization, particularly in the domain of knowledge work.

Some parts are prescribed, others created, and others discovered. How well-proportioned or closely-mixed those three are varies based on the group and may change over time within a group.

The parts that are prescribed to you probably consume most hours on most jobs. The group needs you to do things: attend meetings, make and execute plans, run the systems, provide and receive feedback, whatever it may be. “Do your job” usually means “do the things we know we need you to do and pay you to do.”

The parts you create give the greatest autonomy and represent the most obvious place to make changes. This is where a person can intentionally, in advance, define the role. Some jobs leave almost no room for this, others actively encourage it, and most are in between. I’m fortunate to be in a place that’s heavily biased toward experimentation.

This overlaps with the aspect of a job with the potential for surprises: discovery. A person being proactive in a role, actively hoping to advance a group’s cause, and willing to step out of their comfort zone will probably discover new options in a role that they can’t imagine in advance. By contrast, someone else might find that their role diminishes because of lack of work, interest, or passion. In an organization that is open, changes rapidly, or is new, discovery may be a large portion of the work.

I’m still working out the balance of these at my current workplace. Open communication can clarify each of these and how they intersect.

Muscles and bicycles

Some skills are like riding a bike: you never really forget.

Others are like training a muscle: if you don’t practice continually, the skill atrophies and it takes extra work to build it back up.

Writing and tech are both more like muscles. My current project is to rebuild and then expand my competence in both.

In practice that means:

  • more reading
  • more writing
  • more coding

I’m keeping it fairly basic for now, but building up intensity as work and interest requires.

Tools I use for work

As a student and then a self-employed person, I went through several phases of experimenting with programs and devices to do my work. These have been the ones that have stayed with me.

Some notes app

I use Apple Notes for on-the-go notes and once in a while move everything over to Microsoft’s more structured OneNote for long-term storage.

Freedom

Usually my self-control is pretty good. When I’m reopening Twitter for the fourth time in an hour and quickly running into the same tweets I just saw, I know I need the intervention of an external force. Three clicks of the Freedom icon on my Mac and I can block Twitter (and any other distracting site, if I need to) for as long as necessary.

A browser

If the Internet is something you use, this is important. I use Chrome but I have Firefox on all my devices as well.

A terminal*

Ever since I learned to use Unix in college, setting up a terminal (or the Windows subsystem for Linux on Windows 10) is always one of the first things I do. My .bashrc has been incrementally expanded and contracted and relocated for a few years now, allowing complete and portable customization of my shell environment.

(*–pedantic: Yes, it’s technically a “terminal emulator“. No, this particular language difference does not matter to me.)

A text editor

For code, short messages, drafts, and occasional distraction-free writing, I always have a text editor. The default programs are fine, but I like Atom on my Mac and Microsoft Visual Studio on my Windows desktop. If I’m running a terminal I use vim.*

*If you’re torn in a Unix environment, just use the one you learned first – vi, vim, emacs, nano, whatever. In almost all cases, the differences are less important than loud people say online.

An office suite

In 99% of cases this is Google Drive. I use it for word processing and for my sprawling budget spreadsheets. Collaboration is easy, the interface is pleasant, and it’s free. Microsoft Office has more features, though, so if I need more tools than Drive supports I turn to Office. Exception: I use Keynote on my Mac for presentations because I like it quite a lot. I seldom use any other presentation tool.

A scanning app

I don’t like to carry around a lot of paper. I digitize my receipts and other paperwork whenever I can. When I had a desk I could use a document scanner, but now I just use Scanbot Pro. Once I have PDF’s and upload them to the cloud, I discard the paper copies. I will almost never need them later anyway, and a few megabytes on a computer is better than several drawers in a file cabinet.

I do keep paper copies of tax documents and papers that are significant as paper artifacts, but everything else gets scanned.

Some bibliographical tool

This I used more as a student (and now faculty member) and I hesitate to include it in a list I’d prefer to be general, but it saves a lot of time. Most people can use Zotero. I work in CS where most people write in LaTeX, so I use the BibTeX format instead whenever I can. Usually citation contents can be directly exported in that format from the site where the source was accessed, but if not the LaTeX world is well-documented and most labels (e.g. “author”) are intuitive.

What I don’t use

I’m not a strictly minimalist person, but a lot of the utilities and helpful programs you can find online aren’t necessary. Investigation, installation, configuration, and maintenance take time that could be better used elsewhere, in return for questionable time savings in the long run. I don’t use a window manager on my Mac, for example, because mostly managing windows on my Mac is fine with the built-in tools.

Being cautious about what I install helps save time and delay the inevitable buildup of system cruft that has to be purged once in a while.

This is it

There are a dozen other tools I’ve used: QuickBooks accounting software, map/direction tools, chat clients, maybe an email client, etc. In specific cases, those are worth exploring and probably worth running. This is a list of what I consider a solid foundation for productive work for most people.

Lessons from running my business

Millions of people are self-employed or run small businesses, and millions do one of those for a while and then move to something else. It’s hard for me to imagine that any of us has learned something unique to us individually.

It’s also incredibly easy to find other writers talking about it, sharing the things they learned, and the lists have a lot of overlap. A lot of the lessons also come across as pedestrian.

Still, I wanted to do a retrospective of my own experience. I wanted to approach it from a somewhat narrower perspective. To compile my list, I asked: What made an impact? What lessons did I learn from starting, running, and closing a one-person business that will cause me to act differently than I acted before I ran the business? Here are some of my answers, which of course I reserve the right to revisit.

People will give way more power to you than they should, and you have to be ethical about that.

I always asked permission before going into people’s files. And yet I was consistently told, “Oh, there’s nothing in there that’s secretive or anything, do anything you need to.” This was stunning to me.

I also saw the downside of this trust, repeatedly: computers bricked entirely by the IT support scam industry exploiting predominantly elderly people with little computer savvy and trusting hearts. On a few occasions I saved the computer only because of the sheer laziness of the scammers: one locked up a Windows computer with a particular encryption tool and set the password to “123456”.

Work honestly, and beware people who are willing to do otherwise.

Sometimes you fail. That hurts but it’s (probably) okay in the end.

This was especially true late in the business, as I was winding down and could see I had too little time to finish everything. I had a few uncomfortable phone calls and text exchanges trying to establish a compromise that met as many of my (understandably) annoyed client’s needs as possible within some tight time constraints. The end results were good for none of us but acceptable for all of us.

You will make mistakes and fall short. And that’s (usually) fine, if you patch it up as best you can, make amends, and learn from the experience.

Location is important.

I did not quit because I didn’t like the work. The job itself was fine. A few projects were great, a few were terrible, and most were in between.

I quit because the place I was living didn’t have what I needed for personal and social fulfillment (I’ll likely return to this topic in the future). I’ve never wanted to define my life around economic considerations alone, and I have the luxury of making that choice. When I got the chance at a job that paid about the same, in a place that in other respects was much better for personal development, I took the opportunity and left.

Coda

I closed down but mostly consider the business a success. I started because I needed to generate money for student loan and car payments. I made some money, got a few nice things, built my confidence, and proved my self-sufficiency. I figured out a few things I don’t want to spend my life doing. I could hardly have asked for a better first job after college.