Signal v. Noise

Can you believe we used to willingly tell Google about every single visitor to basecamp.com by way of Google Analytics? Letting them collect every last byte of information possible through the spying eye of their tracking pixel. Ugh.

But 2020 isn’t 2010. Our naiveté around data, who captures it, and what they do with it has collectively been brought to shame. Most people now sit with basic understanding that using the internet leaves behind a data trail, and quite a few people have begun to question just how deep that trail should be, and who should have the right to follow it.

In this new world, it feels like an obligation to make sure we’re not aiding and abetting those who seek to exploit our data. Those who hoard every little clue in order to piece of together a puzzle that’ll ultimately reveal all our weakest points and moments, then sell that picture to the highest bidder.

The internet needs to know less about us, not more. Just because it’s possible to track someone doesn’t mean we should.

That’s the ethos we’re trying to live at Basecamp. It’s not a straight path. Two decades of just doing as you did takes a while to unwind. But we’re here for that work.

Keep reading “The last tracker was just removed from Basecamp.com”

We just published “The Basecamp Guide to Internal Communication“. It’s a collection of philosophies and day-to-day practices that help guide the way we communicate with each other at Basecamp.

We cover when to write stuff up in detail vs. when to chat about it. Why meetings are a last resort, not a first option. How companies don’t have communication problems, they have miscommunication problems. Why a single central source of truth is better than different versions all over the place. Why writing benefits everyone, but speaking only benefits those who were there.

We hope you find it useful. Check it out.

I’ve heard this one before. I’ve used this one before.

“No one’s complaining” so it fine.

“No one” really means “no one has complained to you“. It doesn’t mean no one is complaining to someone else, somewhere else.

In fact, if the thing you make/sell isn’t meeting someone’s expectations, there’s a good chance you’re the last one who’d hear the complaint.

Contacting the company to complain is pretty far down the list. At the top are friends, family, colleagues. If you aren’t hearing the complaint it’s likely because it’s directed elsewhere. People typically talk truth behind backs, not to faces.

Reputation erodes in the shadows before it comes to light.

It’s not all that different from a manager or CEO eventually discovering something was wrong but “no one told me sooner”. The higher up you are, you’re often the last to know.

Out of everywhere someone will complain, you’re close to nowhere.

“I haven’t heard anyone complain about that to me” is a more accurate statement.

So next time you say “no one’s complaining” you may be right, but you’re probably wrong. Doesn’t mean you need to do anything about it – not all complaints are worth acting on – but it should serve as a reminder that there’s a lot you don’t know.

Back in October, we announced our own two-factor authentication solution, dropping the requirement for having a Google account to benefit from this necessary level of protection for your account. This solution is based on TOTP (Time-based One Time Password Algorithm): you configure a special authenticator app with a secret we provide and then your app generates codes depending on the time that we can verify on our side after you have entered your credentials, as the second step before login you in.

This kind of second-factor authentication is very convenient and easy to use, but it has weaknesses. A sophisticated phishing page could trick you into entering your username and password and then your second-factor code and use it right away to log in into your Basecamp account. This is where FIDO2: Web Authentication (WebAuthn) comes in. I’m happy to announce we now support WebAuthn for security keys and other authenticators as an alternative 2FA method. WebAuthn is the newer standard for secure authentication on the web and is more widely adopted and supported by browsers and authenticators than its predecessor, U2F.

With WebAuthn we can offer a 2FA method resilient to fancy phishing attacks because authentication relies on public-key cryptography to prove to Basecamp that it’s indeed you who is logging in, and this proof only works for https://launchpad.37signals.com/ and not for a fake phishing page that copies perfectly our own Launchpad.

WebAuthn is currently supported by all modern browsers in desktop and mobile platforms:

• Desktop: Chrome 67, Firefox 60, Opera 54, Safari 13, Edge 18
• Mobile: Chrome for Android 78, Firefox for Android 68, Safari for iOS 13.3

Another cool aspect of WebAuthn is that it opens the door to new kinds of authentication devices and you no longer need to own special hardware keys. You’ll be able to use your phone or laptop and authenticate with a PIN, or use a fingerprint reader or facial recognition. For example, you can use your Android phone’s fingerprint reader in Chrome, Apple’s Touch ID in Chrome for macOS, and facial recognition, fingerprint reader or PIN via Windows Hello in Edge. Of course, you can also use specific security keys like Yubico’s YubiKeys, which is what I use. Older keys based on the U2F standard work too as WebAuthn is backwards compatible with U2F authenticators, so if you have one of these, you can register it in your Basecamp account as well.

Read more about how to register your security keys and start using this as your second-factor step for your Basecamp account, and, if you haven’t already, please, please enable 2FA for your Basecamp account now.

---

🔐👩🏻‍💻Implementing WebAuthn and 2FA for your own application

Providing your users with modern authentication mechanisms is a reasonably easy task thanks to a variety of open-source libraries in multiple languages. For WebAuthn, we relied on the excellent webauthn-ruby by Cedarcode. They also have a great Rails demo app to show how to use this gem. I encourage checking this out if you want to support WebAuthn in your Rails app, it’s super useful. For the client-side code, we ended up using the webauthn-json wrapper by GitHub that handles all encoding, decoding and building the different JSON objects.

We’ve got questions by some people about how we implemented our 2FA solution based on TOTP, and this is also fairly easy! We again took advantage of an open-source gem, rotp by Mark Percival, although initially, we had implemented the algorithm described in the RFC for HOTP and the TOTP extension. It’s simple and totally doable if you prefer not to use an external library. If you do this, I recommend testing your implementation using the provided test vectors. We’ve also followed the security considerations detailed in the RFC:

• We store secrets for TOTP (and recovery codes) encrypted using AES-256-GCM.
• To account for time drift between the server and clients and transmission delays, we accept one code generated in the time window before and after the current one and use the recommended time-step size of 30 seconds.
• We need to ensure a one-time only use of a code, which means we can’t accept the second and subsequent submissions of a code within the same time window. We use Redis to “quarantine” a valid code after it’s been accepted and take advantage of its key expiration mechanisms,

I hope you find this useful!

We spend about $3 million every year to run all the versions of Basecamp and our legacy applications. That spend is spread across several on-premise data centers and cloud operations. It does not include the budget for our 7-person strong operations team, this is just the cost of connectivity, machines, power, and such.

There’s a lot of spend in that bucket. The biggest line item is the million dollars per year we spend storing 4.5 petabyte worth of files. We used to store these files ourselves, across three physical data centers for redundancy and availability, but the final math and operational hassle didn’t pan out. So now we’re just on S3 with a multi-region redundancy setup.

After that, it’s really a big mixed bag. We spend a lot of money on databases, which all run on MySQL. There’s ElasticSearch clusters that power our search. A swarm of Redis servers providing caching. There’s a Kafka pipeline and a Big Query backend for analytics. We have our own direct network connections between the data centers and the cloud.

Everything I’ve talked about so far is infrastructure we’d run and pay for regardless of our programming language or web framework. Whether we run on Python, PHP, Rust, Go, C++, or whatever, we’d still need databases, we’d still need search, we’d still need to store files.

So let’s talk about what we spend on our programming language and web framework. It’s about 15%. That’s the price for all our app and job servers. The machines that actually run Ruby on Rails. So against a $3 million budget, it’s about $450,000. That’s it.

Let’s imagine that there was some amazing technology that would let us do everything we’re doing with Ruby on Rails, but it was TWICE AS FAST! That would save us about ~$225,000 per year. We spend more money than that on the Xmas gift we give employees at Basecamp every year. And that’s if you could truly go twice as fast, and thus require half the machines, which is not an easy thing to do, despite what microbenchmarks might delude you into thinking.

Now imagine we found a true silver bullet. One where the compute spend could be reduced by an order of magnitude. So we’d save about $400,000/year, reducing everything we spend running our app and job servers to an unrealistically low $45,000/year. That reduction wouldn’t even pay for two developers at our average all-in cost at Basecamp!

Now let’s consider the cost of those savings. We spend more money on the 15-strong developer team at Basecamp than our entire operations budget! If we make that team just 15% less productive, it’ll cost us more than everything we spend to run Ruby and Rails at Basecamp!

Working with Ruby and Rails is a luxury, yes. Not every company pay their developers as well as we do at Basecamp, so maybe the rates would look a little different there. Maybe some companies are far more compute intensive to run their apps. But for most SaaS companies, they’re in exactly the same ballpark as we are. The slice of the total operations budget spent running the programming language and web framework that powers the app is a small minority of the overall cost.

For a company like Basecamp, you’d be mad to make your choice of programming language and web framework on anything but a determination of what’ll make your programmers the most motivated, happy, and productive. Whatever the cost, it’s worth it. It’s worth it on a pure cost/benefit, but, more importantly, it’s worth it in terms of human happiness and potential.

This is why we run Ruby. This is why we run Rails. It’s a complete bargain.

I saw the Senior Programmer offer one day before going to bed. I decided I wasn’t going to apply. I had tried four times since 2013, and I never got to pass the first filter. Each attempt took me a good amount of time and energy, and I didn’t want to go through that pain again. That feeling didn’t last much: the next day, I was already working on my application. 

I knew I had made a mistake in the past: a too long cover letter. So this time I decided to fit it into two pages:

• The letter itself on one page.
• A distilled list of my relevant projects and articles on another.

The self-imposed two-pages constraint was arbitrary but served its purpose: reducing my big initial dump of assorted ideas and projects into something essential and easy to digest.

I decided to add a third page with a “Basecamp timeline” of my previous attempts. I wanted to highlight my genuine motivation, as well as referring to a demo I prepared years ago. I considered this secondary, so I placed it as a light appendix at the end.

I tried to make the application look nice with my limited design skills. They asked for a PDF, but I didn’t want to deliver a boring document. This was the version I sent.

They liked the application, and I was asked to do a technical exercise. Not too big, but interesting and fun. I could solve it on my own terms, and it let me show some technical and communication skills. Then I had three interviews: one with Andrea, the Head of People Ops; another with members of the team: Rosa, Jane, and Justin; and a final one with Jeremy, the team lead. Each interview left some candidates out, and I always knew how many of us were left.

Interviews were not hard. No difficult questions or puzzles. They felt like chatting about my background and Basecamp with colleagues. They tried to make me feel comfortable and gave me their full attention, which was a big contrast with other past experiences I have had. I listened to this episode about how Basecamp hires many times during the process, and it is a pretty good depiction of what to expect.

Because I imagined how good other candidates would be, I always thought it wasn’t going to be me. At first, that worked as a self-defense mechanism against the likely rejection. But the final interviews, when I knew I had a chance, were incredibly nerve-wracking.

I hate describing rare fantastic outcomes as just the logical consequence of some actions. I am proud of the job I did but, when it comes to selecting people, there is no best or right, and I was very lucky to be the one. Said that, I hope sharing my cover letter and experience can inspire other future candidates, like I was inspired by others Basecampers before.

You’re bound to make mistakes as a new manager – but here are the biggest, most common pitfalls to avoid in your first 30 days as a new manager.

I’ve never quite known the proper word to describe the feeling of being simultaneously elated and terrified — but your first 30 days as a new manager is that feeling.

You don’t want to mess this up. You’ve been reading The Effective Executive and High Output Management, googling “management 101” and “first 30 days as a new manager”, and talking to mentors about the “should’s” and “should not’s” of leadership… all in hopes that you won’t make any egregious blunders during your first month on the job.

But quite frankly, it’s bound to happen. You’re going to make a mistake, or two, or twenty. When we’re new as leaders, we operate out of instinct. It’s an instinct formulated from what our former bosses have done, honed by our own value system of what we personally prefer, and our best guess for “So I think this will work?” But we don’t really know if it’ll work.

That’s where I can help 🙂

Keep reading “6 mistakes to avoid during your first 30 days as a new manager”

What should you do for your end-of-the-year review with an employee? Use this one-on-one meeting template.

With December upon us (already!), many managers have been asking me if I have a one-on-one meeting template for their end-of-the-year review with a direct report.

Yes, I do have one 🙂

The end of the year is an opportune time — an ideal time, truly — to reflect together with your direct report, on what went well, what didn’t, in what they were most encouraged, and in what ways they weren’t.

Keep reading “The one-on-one meeting template for your end of the year review”

Get out your Bunsen burner! It’s time to do some experiments. In the latest episode of the Rework podcast, we talk to two businesses that aren’t afraid to try new things. First, the three founders of The Mad Optimist, a soap company in Indiana, talk about letting customers choose what they pay for their products. Then Natalie Nagele, the co-founder and CEO of software company Wildbit, talks about an ongoing experiment with four-day work weeks and what she’s discovered about productivity, happiness, and deep work.

Dave Teare is the co-founder and official “heart and soul” of 1Password, which recently raised $200 million in its first round of venture capital. Basecamp is a longtime happy customer of 1Password and also a longtime critic of venture capital, so the funding announcement led to some back-and-forth on Twitter between Basecamp co-founder David Heinemeier Hansson and Dave Teare. In the latest episode of the Rework podcast, DHH and Dave get on the phone to hash out their feelings about venture capital and what this funding round means for 1Password’s future. (A transcript is also available on the episode page.)

If you’re new to Rework and enjoyed the conversation between Dave and DHH, be sure to check out this episode where DHH and Automattic’s Matt Mullenweg get on the phone to discuss power in open source communities. And subscribe to Rework via your favorite podcast app so you get our new episodes as soon as they’re released.