Tag Archives: IT

Confessions of a sysadmin

Ah, how well I remember 9/11.  That fateful day we sat in the office, slackjawed, watching those planes crash into those buildings, on repeat.  That day I realized my boss was visiting family in New York that weekend and he was stuck there in a no-fly country for the remainder of the week.  Thus leaving me, a junior network admin with only six months experience, in charge of an entire company’s IT backbone.

Thus began my first serious job in a 16 year career as a sysadmin.

We’re kind of the forgotten stepchild of the internet boom.  We’re not rockstars on top of the latest startups and coolest apps, but we’ve been there from the beginning, right in the middle of it all.  I’ve been in long term jobs and short term, seen large companies and small, perfectly choreographed IT departments and chaotic messes in permanent panic mode.

I’ve learned a few things over the years, and I feel like I need to write this just to remind the world how underappreciated we are, in every sense of the word.  And it’s this underappreciation that is costing companies a lot of money in productivity and reputation.

Of course I write this as the corporate world is reeling from the WannaCry ransomware attack.  As I see the headlines scream, I wonder how many of these companies realize their primary weakness is a cheapskate IT budget.  Hackers know this, it is what they seek.

See my view of proper security is a lot like my taste in coffee.  You don’t need the fancy stuff, you really only need to do a couple simple things right, and do them consistently, to keep the final product fresh and productive and not a nasty sour mess.  These hackers aren’t looking into some super complex Hollywood style hacks.  They’re looking for a few simple gaping holes that companies tend to have and exploit those.

And I’m not even going to get into those loopholes.  Because look at it this way.  The companies that got hit the worst by this WannaCry virus probably have nobody in the company who would ever read an article about the WannaCry virus.  Imagine what kind of out the box solutions these people would put together, and leave on all the defaults, or some boneheaded password?

In a world where everyone is facing the public, imagine how many potential tagets a hacker has out there to prod and poke for easy prey?

But hey, it works now, what do I need to worry about?  This company made it another day, what do I care about a possibility 90 days down the road?  Well, sure, if you’re a company that plans to only be in business 90 days, you don’t need to spend a lot of money on IT.  And it’s someone like me who could tell you that, not a consulting firm that sells you costly ineffective solutions to pad their overhead.

But if you want to be in business longer than 90 days, you probably need a sysadmin, at least part time or on some consulting basis, to tell you what you’re doing wrong and what you need improvements on.

And this is probably a good segue into the hopefully most long-lasting business of all, our government.  Hillary’s e-mail scandal was a eyebrow raiser for I’m sure a lot of us in the sysadmin world.  First off, it’s never an e-mail server.  At least in the private sector we deal with a whole system of servers, services, appliances, to keep in compliance with audits.  What kind of government do we have where a major politician can have her own e-mail “server” and nobody notices?

I don’t even care what’s on the e-mails, really.  That was a side issue to me.  But it sounds like we have to have a major popular audit of our government’s systems.

Another case in point – the city of Los Angeles was flirting with free citywide wifi.  They estimated the whole project at … $5 billion.

$5 billion.  For wifi.

You know, I don’t even care.  It doesn’t even matter that my friend and I brainstormed a way to take two zeroes off that price and still come out rich.  I’d just like to know, just whose ankles out there are getting broken to keep that government contract so lush?  Really, it’s okay.  I won’t tell or raise a stink.  I’d just like to know.  Who committed suicide by shooting himself in the back of the head for this?

And yes, I have worked and applied for jobs related to the city.  Let’s just say as far as the city’s concerned, it’s obvious they’re just one big ring of nepotism.  See the way you apply is, says the clerk in charge of hiring, is you find someone we know, and tell them about yourself, and if he’s high up enough in the pecking order he’ll get you this job.

It’s my hope that with Trump in office the pecking order will be a bit upset, and maybe some fresh blood can get in here.  I’m not keeping my hopes up, but hey, he hasn’t disappointed ME yet.

Oh, and then there was the time Obama (and alll those biiig internet companies) thought that kid had a clock science experiment and not a cheap stunt to get national attention.

On that note, then there’s companies working with the government.  You’d think a company that works on our space program would do their best to hire the best sysadmins to work with the best engineers, right?

How silly of you.

Here’s how my tenure at this job ended:

“What’s the password for liptontea?”
“Liptontea IS the password.”
“….you know what?  Go home, hand me your badge.”

Now I know you’re thinking, there has to be some context to this.  Surely I must have been caught peeing in the middle of the office just before that.  Rest assured the context is exactly what it sounds like.  He was struggling with a systems problem (that I was hired to manage), I answered him with an answer that confused him, and rather than deal with his panic and confusion decided to fire me on the spot to discharge his frustrations.

And no, this wasn’t an isolated incident.  Problems that would have taken 30 minutes for me to solve (and were solved) were hacked away at overnight in a total chopped-off-head panic.  Nevermind how much you’re paying me.  How much are you paying these stooges to solve your problems?  Whose kneecaps got broken to keep these guys their jobs?

Did you look past your other employees long enough to ask me a couple questions?

How many people can you replace with one good sysadmin?

A sysadmin isn’t just the computer monkey that makes things magically better when they go wrong, though you’re certainly welcome to think that.  We’re almost a part of management – really, we would be, but we don’t want to be.  But we make policies and systems that go best with the needs of the company, and we train people to work effectively with the system we design.  With us at the helm, people work happily, effectively, understanding and having a part in the systems they work on all day.  We reassure the paranoid, and warn the careless.  You’re content in knowing you have someone who keeps the internet carpetbaggers at bay, like all those guys selling you THE CLOUD.  And in knowing we’re anticipating problems you never heard of.

Like WannaCry.

BTW in the articles I read about WannaCry they never talk about backups.  You do know backups are a great defense against ransomware, right?  How are your backups, anyway?   I ask this fully knowing most companies you think would be automatic about this, have at least some problem with their backups.

Like I said, I’m not asking you to spend money where you shouldn’t be.  If that was my aim, I wouldn’t be writing this article.  I’m just saying, you have a limited amount of money, spend it wisely.  We’ll tell you where it should be spent and where it’s wasted.  But cheap out on us, and you’ll be penny wise, pound foolish.