Datacentre Trolley of Awesome™

It’s been two years since I posted something on this blog. That is fairly abysmal in itself but let’s not focus on the negative things. I’m back! Sort of! Probably. It’s been a roller coaster couple of years for me with lots of really fun and exciting things going on and a few not so fun. However. I’m not here to write about any of that! No. I’m here to write about something far more mundane …

The affectionately penned #DCTrolleyofAwesome on Twitter made it’s appearance a week or so ago but in reality this little side project has been bumbling around for a few months now in the background at iomart’s DC5 in Maidenhead. I’m writing this post to mainly consolidate my thoughts on the entire process but I’ve also had a few people reach out from Twitter asking to get a little more detail than just the photos which I was taking at the time. So here we go. Strap yourself in to be thoroughly bored 😉

The Problem

DC5 is by no means the biggest datacentre in the world. It’s not even the biggest in the UK. However the entire site is still just over 50,000 square foot of space. Not inconsiderable and on the inside, it’s a pretty busy place. Reboots, OS Installations, Racking, Cabling, PDU and Air Conditioning checks … the list of tasks is endless and many of them require a reasonable amount of walking from an engineers desk.

The real issue, is efficiency. Most tasks have a ticket that need to be updated. Or a system that needs to be checked. Or a colleague that needs to be advised about the outcome of something. It’s grossly inefficient for one of our datacentre engineers to wander back to their desk, do the Thing, and then perhaps have to wander all the way back to the opposite end of the datacentre.

So – we needed something a little bit smarter that would let our engineers work from anywhere in the Datacentre, without the need to traipse back to their desk.

Potential Solutions

Use your phone!

Yep. Good for maybe one or two of the tasks but even the best mobile optimised web interfaces would struggle with the sheer volume of information that our engineers need from time to time on the datafloor.

Use a tablet or laptop!

Mm. Good but not great. Attaching and carrying around barcode scanners, label printers, cables for powering servers … far too cumbersome.

Do you really need this? How much time is it saving, really?

I pondered this myself more than once. Rather than relying on my gut feeling, and more than a fair amount of feedback from our engineers I decided that some good old fashing dogfooding was required. That’s dogfooding. Like, Eat Your Own Dog Food. No, not … get your mind out of the gutter!

I spent a couple of days doing the job of our datacentre engineers equipped with a stopwatch. In the couple of days I kept track of how much time I spent walking back and forth to my desk – I clocked in just over an hour each day of walking. Completely unproductive, zero output, walking. Other than improving my waistline I suppose? Extrapolated out, it’s a huge amount of time every year where we could have been helping our customers by delivering better, more efficient service.

Re-inventing the wheel

I really didn’t want to do this. I looked at dozens of other options before deciding that our own home-baked solution would work out better. Some of the things we considered:

Medical type trolleys – extremely modular and ticked a lot of boxes, unfortunately we found as the companies sell in bulk to healthcare and we only needed a few the costs were astonishingly high for what it was whilst still not being “perfect”.
The Big Blue F Option – various Google searches stumbled across this bad boy from Facebook. More about this next as it’s moderately interesting.
Traditional stands – you know the type, tall, monitor at the top, really wide wheels. Not ideal – especially if you need to work in a narrow hot aisle.

What was left? DIY! Just before we get on to that though …

The Facebook Crash Cart

Someone must have built something like this before. Surely. Right?! Right.

Sadly details on the particular construction of this fella were pretty sparse. I even know an Infrastructure dude who works for Facebook and was initially met with an “ummmm” – which is pretty unusual for Facebook given that they tend to Open Source nearly everything they do in relation to a datacentre via The Open Compute platform.

Stumbling around Reddit I came across this thread: This Datacentre Crash Cart is a thing of Beauty

In the depths of that thread, is a comment here, by the supposed inventor of said thing of beauty. Unfortunately for whatever reason his website has been removed and not in the mood to wait for a response by email we forged ahead with our own setup.

What did we need it to do?

On paper (ok fine, Notepad …) we wrote down the main things we wanted to achieve with this side project.

Portable – it has to be the right size to hold a 2U server on top of it, whilst also being neat and nimble enough to get down the hot aisle of a datafloor without drama.
Powered – we didn’t know yet how we were going to do it, but we knew we wanted some sort of workstation on there, maybe a laptop or a small desktop.
Printer – asset tags. Lots of them in our datacentres. Being able to print them on the move was crucial.
Switchable Monitor/Keyboard – traditionally these stands were used for checking the output on VGA from a server, resetting passwords that sort of thing. We didn’t want to lose that.
Barcode Scanning – asset tags again, need to be able to scan assets and find them on our system!
Wifi Capability – because the datacentre is big and really long ethernet cables are not cool or clever.
Session Persistence – we weren’t sure how yet, but the answer came quickly but we knew we wanted some way of an engineer being able to move between trolleys or even back to their desk and just to carry on working.

Not a huge list of things, but a few challenges in there for sure. So, before we break down the detail – how did we do?

What we ended up with

This is the first trolley of six. A few things we can improve upon certainly but broadly speaking we’re pretty happy with it!

The Details

All photos are linked if you want to view them fullsize 🙂

The Trolley

The Luxor Automotive Utility Cart! Luckily we actually had one of these already to play with. It’s the perfect size and is really sturdy with good quality wheels. You can buy them online from a few different places. Just be careful to get them with the three tray variant – not the two, which is nowhere near strong enough.

Cabling & Power

Our “beta” version had all the cables loosely bundled with velcro around the outside of the frame. We weren’t happy with this at all since it was messy and didn’t look great. On the finalised version we’ve drilled holes in the arm under the monitor. Just big enough to feed cables through without affecting the structural integrity of the trolley. This keeps things – broadly speaking – neat and tidy.

It’s worth noting too that we had to convert the power cables for a couple of our accessories from 3 Pin plug to IEC capable. Not a big job but something to bear in mind.

We had some old horizontal IEC power bars laying around that conveniently fit perfectly inside the trolley tray. These are generic rack width PDU’s so you can get whatever variation you prefer!

The UPS again – we had these laying around and are definitely a nice-to-have rather than a must-have. If you’ve got them them, why not?

VDI and the Dell/Wyse Wifi D10DP Thin Client

So. Laptops. Phones. Tablets. No thanks.

We wound up with a Dell Wifi D10DP Thin Client. I was a little skeptical at first so our friends at Dell arranged to ship us a Demo unit and I’ve been blown away by it. It’s lightning quick to boot and Just Works. We’ve hooked it up to the iomart Corporate Virtual Desktop Infrastructure running on VMWare Horizon View (which is fantastic btw!) and it’s given us complete automation and movement without a major cost point.

The VDI and Monitor are mounted to a monitor stand which is drilled and secured on to the top shelf of the trolley for stability.

Printing

Relatively simple! The Dymo TwinTurbo has been a staple for us when printing asset tags for a long time now. The Thin Client picks it up painlessly when connected via USB.

Switchable Keyboard/Monitor

We looked at a few solutions for this, eventually the cheap and cheerful one came out on top. The ieGeek VGA Switch is a lovely little bit of technology and the only adjustment we had to make was slightly longer screws to hold it to the trolley in the original plastic casing. One press of the little silver button hops you between the VDI Thin Client, and whatever server/device you’ve connected the second cable into.

Keyboard/Trackpad

This was, believe it or not, probably the trickiest part. We knew we wanted the keyboard to be at the “end” of the trolley, as having it at the side when working in a hot aisle wouldn’t be convenient. That meant, because of the width of the trolley we needed a keyboard / mouse combination that was really narrow. We settled on this little number from Keysonic via Scan. That sorted one problem.

The next was that it seems nobody wants to make slideout keyboard … things … for little keyboards. So we had to Macgyver it. We bought a normal sized slide out keyboard tray, and threw away the tray bit. We kept the rails and just attached it to a spraypainted slab of MDF. Once mounted, it looked and works the absolute business! You can see the keyboard in the first picture of the trolley in this post, but on the underside it looks like this (probably need to click on this one to see it clearly).

We also added a few small self-adhesive C-Clips to keep the keyboard cable neat and tidy.

In Summary

Still with me? 😀 Well done!

This has been a fun side project to work on which I believe holds real, tangible value within a Datacentre. If you’re only dealing with a dozen or so racks, it’s definitely overkill. When you’re into territory of 700+ (and rising) racks in a single facility, spread across two floors and 12 different data halls? At that point it just makes sense.

I did consider maybe shoving this on Kickstarter or something silly like that, but instead we’ll keep it simple …

Datacentre Trolley of Awesome by Johnathan Williamson is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based on a work at www.suitedupgeek.co.uk.